{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_PGI_Rvgr0bo"
   },
   "source": [
    "#  Training a Generative Adversarial Network on MNIST\n",
    "\n",
    "\n",
    "In this tutorial, we will train a Generative Adversarial Network (GAN) on the MNIST dataset.  This is a large collection of 28x28 pixel images of handwritten digits.  We will try to train a network to produce new images of handwritten digits.\n",
    "\n",
    "\n",
    "## Colab\n",
    "\n",
    "This tutorial and the rest in this sequence are designed to be done in Google colab. If you'd like to open this notebook in colab, you can use the following link.\n",
    "\n",
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/deepchem/deepchem/blob/master/examples/tutorials/Training_a_Generative_Adversarial_Network_on_MNIST.ipynb)\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 188
    },
    "colab_type": "code",
    "id": "cyXeZ5zTFkah",
    "outputId": "521d8d0b-3bbd-41ef-cb5f-06587d2679f8"
   },
   "outputs": [],
   "source": [
    "!pip install --pre deepchem\n",
    "import deepchem\n",
    "deepchem.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "06xelFpir0b6"
   },
   "source": [
    "To begin, let's import all the libraries we'll need and load the dataset (which comes bundled with Tensorflow)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "23zZTDoar0b7"
   },
   "outputs": [],
   "source": [
    "import deepchem as dc\n",
    "import tensorflow as tf\n",
    "from deepchem.models.optimizers import ExponentialDecay\n",
    "from tensorflow.keras.layers import Conv2D, Conv2DTranspose, Dense, Reshape\n",
    "import matplotlib.pyplot as plot\n",
    "import matplotlib.gridspec as gridspec\n",
    "%matplotlib inline\n",
    "\n",
    "mnist = tf.keras.datasets.mnist.load_data(path='mnist.npz')\n",
    "images = mnist[0][0].reshape((-1, 28, 28, 1))/255\n",
    "dataset = dc.data.NumpyDataset(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qijPRZXOr0cI"
   },
   "source": [
    "Let's view some of the images to get an idea of what they look like."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "mmhulNHor0cK"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC0CAYAAAAn8ea8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOydeXSb5ZX/P9p325JtyZa3eI3teItxEsdJnJ0SAjTQAaaUlna60M50Yzp02sPMdJuWmZ4e6HRhzmEKtKVACiUQKKGEhBCyO4vtxLEd7/tuy7Yky5Ylvb8/8tPbmCRksRYn1eccHYIk6716dd/nfZ773Hu/EkEQiBDhZkUabgMiRAgmEQePcFMTcfAINzURB49wUxNx8Ag3NfJrebNEIlnQIRdBECQQsTOAjAiCEA8L31b/Of0wkRE8wkfRGW4D5kvEwSPc1EQcPMJNTcTBI9zUXNMi828ZtVqNVqvFarUSFxeHIAiMjo7S19fH1NQU09PT4TYxwiVYcA4ukZxfDPtzZPz/7ydcuTNms5mcnBw+9alPsWnTJgD27NnDH/7wB5qamuju7g6LXdeDRCIJ23m8Gi78zedrp+RaPiAQoSKpVIpSqUSlUonPRUVF8fGPfxyFQoHBYMBisfDLX/6S9PR0br/9dtGhxsbG2L59O6+//volHSoY4TepVEp2djZf+tKX2Lp1KxaLBa1WC8DU1BQDAwP8/ve/5/HHH7/qzwxXmDAhIYG8vDy+973v8eKLL7Jr1y56eno+6k9OCoJQBqGxNSUlhbvuuouHHnqImJgYpqen+epXv0pjYyNDQ0Mf+beXCxMGbQRXqVQolUqkUilGo5Ho6GhMJhNqtZq4uDgSExPF92q1Wm655RaUSiVyuRylUsm6detIT09n1apVpKenY7fbcblcTE9P4/P5gmX2HDQaDbGxsWzbto3ly5eTmpqKUqnE4/Hg8/mQy+UkJyeTkpKCxWJhZGQEn883r1FHpVIRGxuLUqnEbrczOjoasO8TFRVFWloaer0etVqNXL5wbuCpqamUlJSwefNmcnNz0el0TE1NodVq52VnUL6hVCrFZDJhNBpRKpXk5uaSlpZGXl4eBoOBzMxMCgsLL/m3U1NT9Pf3c+utt2K1WsnOzmZiYoKenh6amppoaGjA6XQGw+yLvkNcXBwFBQV88YtfxGKxoFKpEASBmZkZZmZmcLlcJCQkkJCQQH5+PqdOnWJqaorZ2dnrPq5Op2Px4sWYTCZaW1sD7uApKSn09/djt9sD9rmBIC8vjzVr1rB582aUSqU4SAiCMK8BI+AOLpVKyc/P52tf+xp33nkner0emUyGRCJBKpWK/70UgiDQ1tbGL37xC9GJJRIJAwMDOJ1OJicnaW5uxuv1BtrsOfarVCry8vLYunUrn/70p0lJSZkzigwMDFBXV8frr7/Of//3f7N161YqKyt58skneeutt6ipqbnu48fExFBRUUFaWhr79++f12ddiEQiwWAwkJqaSlJSEgaD4bK/QziIi4vDbDajVCoD+rkBd3BBEJiYmGBmZgaJRIJOp7vs+7q7u5mZmUEqlbJo0SK8Xi9DQ0McOHAAj8cDgM/nw+Vy4fF4cLvdQXVuAKvVym233UZlZSW5ublYLBZkMtmc98TFxREbG4sgCHR0dCCVSomJiRFv//PBaDRSUVHBxMREQB3Qf+csLy+nt7eX/v5+RkZGAvb514tSqRSnosXFxQCMjIxQU1PDvn37aGlpmdfdJmgO3t/fT29vL2q1GrfbjUKhIDo6GgCv18v09DTV1dXY7XakUik+nw+NRsPAwABNTU2BNuuqUCgUWK1Wtm7dysaNG9HpdAiCgMvlwu124/P5MJlMGAwG9Ho9Xq+X5uZmNBoNRqMRk8mERqOZlw3+KUp1dXWAvtV5UlJSyM7OJicnh1OnTjEyMsLk5GRAj3Gt+H1i2bJlLF26lIyMDAC6u7v54IMPeP755xkbG5vXlC8oc/DJyUl27dpFd3c369ato7W1leTkZL785S8DYLPZaGpq4uGHH2Z8fFyMVNx9991hPelpaWksXbqUtWvXio7q8/morq6mpqaGqakpHn30USYmJujr66O+vp7JyUncbjcFBQUXhTSvF/9ULlBIJBIeeeQRVq5cic/no6ura0HMwVNTUyktLeXxxx/HZDKJc+9XXnmF/fv309/fP+9jBG0Z3drayvDwMI2NjdhsNrKzs1m0aBGVlZW0tLSwY8cO7Ha7OBVpb2/n+eefD/oU5FIoFApSU1P5xje+QUVFBTqdDolEwuDgIOfOneM///M/UavVxMfH89xzz3Ho0CGamppobW2lt7eXwsJCJBIJZWVlFBQU0NTURGfntecpJSQkkJKSgtFoDPj8WKfToVKp8Hg81NbWMjw8HNDPvx7y8vKorKwkNjYWhUKB0+mktbWV9957j+bm5oAcI2gO7nA4cDqdjI+PMzMzg0qloqurSwyx+R/+FbLT6QxJdOTDKJVKoqOjKSwspKysjJycHHw+H9PT0/T29lJbW8uJEyeIi4tjfHwcl8vF8ePHGRgYYGpqCofDgcPhQCKREB8fT3Z2NosXL75uB7darajV6oCN4BKJBKVSiV6vF/ce+vv7w3KuL0ShUJCenk5RUREqlQqv18vY2BgnTpygu7s7YHfyoAZCBUFgamoKAI/Hg8PhwOfzkZaWxqZNm3jmmWeYnZ0N665aTEwMWVlZ3HvvvaSlpSGXy5mYmGB4eJjq6mr27duH0+lkYmKC1tbWj/wslUrFLbfcwtTUFLt3775mW3Jzc8nLy0OlUgXMwRUKBSaTCYvFQlRUlJhiEM7UAolEQnR0NCUlJaxevRqfzyeO3tu3bw/o9Clkkf6uri5++ctfkpWVRUFBAStWrGDlypXU1dUFZK51PSgUCrZs2cIdd9zB5s2bkUqlnD59ml//+tfU19fjcDiuOXITHR2N2Wy+ru1wf0wdoKOjIyDTiIyMDL785S+TkZGBz+eju7sbh8Mxr4XbfNDpdCQmJvL4449TVlYmPr9nzx727t3L0aNHA3rxhczBZ2ZmGBwcZM+ePUilUtavX8+WLVtISkqis7OT0dFR2tracDgcIbFHIpFgNptZvHgxhYWF6PV63n//fQ4fPkxVVRX9/f34fD6kUuk1OapMJrvunTe5XI5cLkcQBAYGBhgfH7/mz5DJZKhUKqxWKykpKRQWFlJeXo5Op6Ojo4OjR4/idDrFtU+oiY6OJisri6KiImJjY/F4PPT19VFVVUVNTQ0ulyugxwuZg/t3AHfu3IlOp2PFihXce++9LF26lPb2durr63n99dfp7OzE4/EEfdoilUpJT08nLy+PrKwsvF4vb7zxBnv27KGxsfGaPy/QCUx9fX3YbLaLnvcvPiUSibiB5p/OKBQK1Go10dHRVFRUUFlZSX5+PiUlJSgUCjo6OnjnnXew2+1hWczLZDISEhIoLi4W1xoOh4Pa2loOHDjA6dOnA37MkCcj9Pb2snPnToaHh3n88cdZvny5GL5KT0/nwIED7Nu3j5GRkaD+CCqViu9+97sUFRXhcrmorq7m6NGj1+3cfie78N/zISoq6qKYulKpJCcnR0yFKCoqwmKxoFQqUavV3HXXXeJOoMfjobW1FUEQcLvdyGQyRkZGOHv2bMhyeT5MZWUld911Fw8++CBqtZqOjg5qamp49NFHGRgYCMq6IOQOLggC/f39HDp0iP/5n/9h5cqV5Ofnk56ezrp164iPj0ej0bB7927GxsYCfssCiI2NJTMzk6ysLKKjo3G73VRXVzMxMXFdP74/X0IQBGw2GwMDA9c1mns8HnHqcNddd1FQUDAnGqNSqcjNzUUikYhJWQqFApfLhcPhoKGhgbGxMXG619fXR0ZGBrm5uahUKhwOhzj1CiX+DZ1Pf/rTlJWVYTAYkEgk2O12hoaGGBoawu12B+XYYUknm5ycxOFw8Morr+B0OpmZmUGj0ZCRkUFMTAyzs7N0dXUFLc/aYDCQnp4u5j6Mj49z5syZa169SyQSYmJi0Ov1YsSop6fnitGWyzE6OipuoS9ZsoS0tLQ54TL/tvb09DQzMzO43W4mJydxOp3Y7Xbq6+vp7u6mr6+PM2fO4PF4mJmZAc5fPFNTU0xMTFyXbfNBo9GQlpbGhg0bSElJmfN9u7u7xUhbMAhbvqTP56OlpYXf/OY37Nixg/Xr1/Poo4+yZMkS7r//flJTU9m+fTu//vWvA35suVyOSqVCLpczPT1NX18fu3btuqbcDH98+fOf/zzr16/H6/Vy/PhxXnzxRd56663rsuv555/n0KFDNDc3s3TpUtRq9UXv6ezs5PDhw7S3t9PZ2Ultba14a//wXWPLli2sW7cOo9FId3d3WJwbIDk5mS984QsYDIY5d4+DBw+yffv2oB477AnB09PTDA8Ps2fPHj772c+KkYu8vDyKioooKCigvr4+aLdVf17M6OjoVYfOZDIZycnJbNiwgfvvvx+LxcLAwACvvfYaDQ0N85pL9vf3s337dv785z9flOTlZ2JiQsyNn5mZuex0KDs7m8zMTABOnDhx3XeW+ZCYmMjixYspKysTL1j/YNDQ0MDAwEBQjx9WB9fr9URFRaHX6zEYDHNGLLlcftkfOJBMTk4yPDyM2+2+4rxZKpWi0+lISkpiyZIlbN68GZPJRH9/P2fOnOH48eMMDg7O62J0uVx0dHRc999fiMFgEDd3zp07R19fX0A+91ooKSmhvLycpKQkZDIZbrebiYkJPvjgA9ra2oK+4RQ2B5fJZCxatIiysjKWLFlCQUGBGCGA82Gy5uZmzp49G9SQYXt7O7W1tVd1DKVSSWZmJvfddx/l5eWsWbOGPXv28PLLL/Pcc88FzcZAUF1dTXt7e8iP+w//8A9s2rRJTCMeHh6mvr6en//855cMgwaakDt4cnIyhYWFVFRUsGnTJiwWC9HR0WKoC87fwkZHR5mcnAyac/sLL3Jzc5menr5iHLu8vJxVq1bx2c9+FrPZjNPp5M033+T73/9+wEbcmxG9Xj8nR76pqYnt27czMTERks2mkDi4SqUiKiqKpKQkVq1aRX5+PkVFReTk5KDT6VAoFACMj4+LGYgHDhzgzJkzQbfNYDCQlJREZWUlHR0d2O123G43cXFxYu2oP+acl5dHfHw8DoeDjo4ODh8+TFdXV9jzqq+ERCIR6zBDhVarZeXKlcTHxyOTycRpm81mC3pV1oUE1cH9o2RsbKwY577vvvtISkrCaDQC51f+/modf+D/T3/6E/v37w9q+EgQBHw+H2q1GqvVygMPPMDu3bvp7OzE4XBQVFREYWEhy5cvJzc3l5iYGORyOb29vdTV1XH69Gnef//9oMTpg0FcXBxRUVEhOZY/fPrQQw9htVrFza/Z2VnGxsbo6OgIWSw+aA6u1+uJiYkRk31KS0tJTU1FoVDMyXXu7u7m3Llz/O53v+PIkSMMDg6GpDTNj0QiwWg08pnPfIbbb78dp9PJ1NQU6enpqFQqZDIZUqmUrq4uOjo6qKqq4vnnn6elpUVM/V3oCIIQsB3WqyExMZGioiIqKyuJiYnB6/UyOzvLH//4R/bu3cvIyEjIMkgD6uByuRydTse2bduwWq1YLBbS0tIoLCzEbDaL+ciTk5MMDg5y4MAB6uvraWtro7a2lsHBwZCMiJOTk7S1tdHV1UVSUhJ6vR6FQkFsbCzR0dHMzs6i0+nENM7Tp0/z3nvvUV1dzeDgID09PUHbeQsGEomElJQUTCZTSI6nVCoxGAwYjUbkcjmzs7PYbDZ27txJbW1tSDMZ5+3g/kY+0dHRGAwG4uLiuOeee0hLSyM+Pl5M/4TzI8nk5CQdHR3U1dWxY8cOGhoa6OvrC6nDOBwOenp6qKmpwePxkJycTHR0NCqVSmwNMTk5yfj4OIODg+zbt4+33nqLkydPhszGQBMfHy/WxAYb/0aaRqNBJpMxMzOD0+nk1KlT11UIMi9b5vsBBoOB3NxcvvCFL7B48WLS09NJTEy86Hbor5J56aWX2LNnD7t372ZqaiosxQ5TU1N0dnbyrW99i8rKStasWcNXvvKVOe957bXXOHjwIAcOHKCtre2GmIpcjlBNTRYi83Zwo9HI6tWrWb16NRaLBY1Gg0QiwWazMTw8TE1NDT6fj/7+fvbv309zczOjo6O4XK6wVvIIgsDY2Bj79++ntraWnTt3znm9r6+P8fFxbDbbDevc586dY9GiRaxZsyakxx0fH6ejo4P6+nrS09PD2n9l3g7uXxmfOXNmzkaC38Fra2vx+XwMDAxw8ODBsI3al8LtdjM4OCgWF99stLW1UVVVRVJSEjU1NSGrnPJ3J9u9ezfp6ekoFApxUAs1IW++GUxuFO2bG8VOQtx8cz5ENHoi/E0ScfAINzURB49wU3Oti8wRFq60XNoF/47YGRhuFFvTLvdCZJEZBm4UO4kIwUa4yVmoI/ZVE3HwCDc1EQePcFMTcfAINzVhr6qPMD8yMzOJjY3FaDTidDrp7OxkaGhI7IfyN8+FXZmu9ACEhfz4W7NTIpEI//7v/y7s379f8Hq9Qn19vfCVr3xFsFqtgbL1RLDOqUQiuarHtZ7TDz8iI/hVolQqUSqVaDQaiouLsVgsGAwG/vSnP2Gz2ULezFKlUmE2m7n11lvJz88HID09nczMTFJSUsLSIuJyaDQaLBYLcrkci8VCTk4O//Iv/4JCobhs4l13dze1tbV873vfY2pq6rozOiMO/hEolUq0Wi1FRUXEx8djNBqJj49n0aJFxMTEoFarGRwc5OzZsyEXzhIEgdnZWZxOp9hbxH8R+ou4w4lfec4vW5ifn49cLicuLo5FixaRnZ0ttoq+FEajEb1ez4YNGzh9+jS9vb3XVQkUcgeXSqXIZDLxcaHgkr8WUxAEZDKZWJAc6nxsqVSKQqHAaDSSlJTEJz/5SbKzs0lJSSEjI2NOjaPD4UCpVNLS0hJSOz0eD3a7nba2tjmN8xcKCoWCRYsWceutt1JSUkJFRYUoK+mvLPookVe/mt0DDzwg5u7fEA6en59PYWEhK1asYNmyZXMq7J999lnOnj2LzWajpKSEnp4ejh07xunTp0PmPFqtluTkZD71qU+xbNkysRuqTCbD6/XS39+Py+VCpVKRnJzMxo0bRW2Ztra2kOW6+3w+Me/6ehrlBxuTycR3vvMdysvLMZvNYpeya6kuUigUbNu2jY6ODrq7u69LFDdkDh4VFUV2djYPPvggWVlZLFq0CIvFwtTUFIODg2g0GtasWcPSpUuZnp7GbDbT1taGVqulvr4+qDWb/nbEq1atIi8vj6VLl1JUVERCQgLR0dHIZDIcDgd9fX08/fTT+Hw+Fi9ezMMPP4zBYBDrUQPdBP+jkEqlqNVq0caFxuzsLO3t7ZSVlYmKF06nk76+Ptrb2y86VwkJCcTFxWG1Wud8jr+F3/VWBYXEwZVKJWazmdWrV7N161axGcz4+DgtLS309/cTExMjNtfxizCp1WqGh4eDXvIkk8mIjY2lsrKSVatWsWzZMrRarTjazM7O0tvby6lTp3jllVdQqVTY7XYefvhhVCoVarU6oMpoV4PfwePj4zEYDOLzfpUHhUIREqWMy+F2u6mrq6OkpERcgNtsNhobGzl16tRFDu4Xqf2wsrS/jcf1hj2D7uAymYysrCw2bNjAj370IzQaDf39/WJ/uvr6egYHB1Eqlfzbv/0b69evp7S0FDjfP7qhoSHoEQqtVsumTZu4++67xYiEH0EQ6Onp4X//93959dVX6e/vJykpKaj2XA0ymUyMTlw4glutVvLz82lubmZwcDBsamqTk5O89NJL7N69W2wX4l8UX6oTmE6nE+XT/VMaj8fD+++/z/Hjx69bNzOoDq7ValmzZg2f+cxnWLp0KSqVivr6evbu3curr75KY2MjTqcTuVzOypUrKSsrIzs7Gzh/tdfX1/Pee+8F3cE1Gg2VlZXiWgDO66UPDw/T29vLs88+S01NDaOjo8D5Fgzx8fFBtelKuN1uRkdH2b9/P1KplIqKCgBWrFhBfHw8Q0ND7Nu3L6xygV6vF5vNJt7ZBEG47G+ZnZ1Nbm4uer1evGMLgsDIyMi8VOGC5uDx8fGkpaWxefNmsUddb28vH3zwAQcOHODUqVNiEWpsbCzl5eWkpqaKt9v6+noaGhro6uoK+gLT6/UyMjJCR0cH4+Pj2O12uru7GRwcpLOzk4MHDzI2NiauA/ztnsOJIAhMT09z/PhxUlNTRQePi4tDEATi4+NFvZ5w2niltZNMJiM+Pp68vDxycnLmaIR6PB7q6+vn1QkraA6+cuVKtmzZwpe+9CUEQaCrq4s33niDn//85/T398+ZUxkMBv7+7/9eDHX5fD5eeOEFDh06FJLGljabjZ///Oc0NzejUqk4efKkqIhwKVlDg8Ewp2NqOHnzzTdJS0vj05/+dLhNuWb8YcP169dz++23U1BQMOei9Hg8vPTSS/PSCw24gyuVSj73uc+xbds2VqxYwezsLHV1dXzwwQc88cQTDA4Oim1zlUolWVlZ3HLLLVitVlQqFd3d3bz55pu8/fbbQe/+78fr9TI0NMTLL7+MRCJhZmaG2dnZy945srKyROWEhYC/yanP57shmvzI5XI2b97Mhg0bKC0tJSsr6yIBBJvNRmtr60cqWFzVsQJhsB+pVIpWq2X16tVkZWWh1Wrp7Ozk7bff5vDhwwwODjI7Oyv2rtu4cSN5eXnk5uaiVqsZGBjgzJkz7Nmzh9HR0ZC2c/N6vRfdLeLi4khKSkIul8+ZR2ZkZBAXFwecb+g+MDDAwMBA2BoELZQ+MxfiV8Pw90S8cM2iVCrZvHkzhYWFLFq0CKPRKAru+hXv2traqK+vZ3p6el7nNeDNN/V6PatXr8ZsNjM9PU11dTWvvPIKdXV1yOVytFqtuIX7ta99jezsbNFZmpqaOHLkCHv27AlbW2L/aOi/u6xdu1bs1gXnnSkvL0+c67a0tHDu3Dm6urrC6mh+J1gIzu4X6EpJSWHDhg3k5+dTXFwsvi6TybjlllsuUpH2er04HA5ee+01Tp48SWNj47wbRQXUwWUyGUqlEqPRiEqlor+/n//4j//AbrdTUFDAxo0bWbduHSkpKVitVkwm05wY94EDB/jggw+C2hf8o/A3EU1NTeVLX/oSpaWlLFmyZE7rYX8awezsLAMDA/zwhz/kzJkzN2x7t2CQkJDAkiVLePLJJ7FarWi12jm/s38Q+TDj4+McP36cl19+md7eXnw+37wv2IA6uM/nw+v14nQ6RaHSxx57DK/Xi1arJSUlheTkZJRKpZjHodFokMvlOJ1Ozp07R0tLSyBNuiL+0SYhIYG77rqL7OxsEhISxKb3LpeLzs5OEhISROFVOD/aSKVSMjMzGR4eZnh4OCzy2AsRiUQittJWq9WXjOZ8ePSG87Hw/Px8cnJycLvdAVmDBdzBZ2ZmaGtrE5UdHnzwQVHBYXp6GofDwfDwMHa7nYSEBPFKHhoaoquri8HBwUCadEXUajVGo5GysjLuuece8vLykEgkTExMMDIyQmtrKw0NDZSWlqLVaudETxQKBcXFxYyNjdHZ2cn4+HjYpgj+nUH/3Uaj0YSt6aXH48HhcNDd3S3uSE9PT885NxdKn/ujUlqtlrS0NNLS0ujr61t4Dj47O8vw8DCPPPIIX/va17jzzjuJjo5maGiIjo4Ojhw5wo4dOxgeHkYQBJ566imUSiUej+eahVgDgUKhYPHixZSXl/OLX/wCqVRKa2srf/jDHzh69ChtbW20t7cTExPDN7/5TaKjo8WNKIVCgclk4otf/CKpqanMzs6ye/duXC5XyKcrH55CaTQaysrK2LVrV0jt8DM0NITNZuMzn/kMmzZtQqFQcOLECdxu90XnJiYmhk984hNs27aNlJSUgA8QAQ8Ter1empqa+OlPf8pzzz0nqglPT0+LIlMGg4GMjAwKCwsxmUxMTk5eFBsPJv5R5b777mPjxo0sXboUqVRKU1MThw8f5rXXXmNgYIDZ2Vni4+N57LHHWLVqFUlJSXi9Xvbv38/ExAQJCQmUlJRQVlaGxWJh1apVHDt2jHPnzuF0OoG/dloNJh+qukKtVrN69Wpyc3NFLfhQ4/F4GBwc5O233xZ16S81p1YqlTgcDlasWDFH5jtQBGWjZ3JykrNnz172df9OYExMDCqVCrfbTVtbW8giJ0ajkdTUVDZu3Mjy5ctJSkqioaGBo0ePcuzYMdrb25HJZCQmJpKbm8u6deswm8243W5qa2vZt28fY2NjWK1WPB4PZrMZs9lMZWUler2etLQ0nE4ns7OzdHV1Bd3B7XY7PT09ovCATCbDbDaTlZVFe3t7WBzcv9N6pcoit9vN5ORk0CQFw1LR44+2+K9ol8vF/v37QyIMKpPJyM/P5xOf+AT33nuv2Jz/qaeeYt++ffT09DA1NUVRURG33nor999/P4sXL2Z0dJS6ujp+8IMfUFNTw+TkJDqdjvfee4/i4mLKy8u59dZbKSwsFBeiQ0ND7N69m927dwf1O3V0dPDOO+/w4IMPzqnmWbZsGTabbUFLr6jVapYtWxY0BbiwOHhfXx8zMzNhCa3dfvvtbN26lb/7u7/D4/HwxhtvsGfPHt577z1xylFRUcGaNWswm81otVp+85vfcOTIEerq6jh37pxou8PhoKqqipqaGnbs2MGBAwcoKysjPT0dgOeff57jx48H/TtNTEyEtNjiUvijUcuWLaO7u5v+/v4rbtT5Zb6/9a1vBa0iKSwOrlariY2NDfm2skQiYdWqVRQXF4shQKPRSFZWFkqlkoyMDJKTk8nLyyMzM5Px8XFqamp49913aWxspK+vb840yufz4XK5cLlc2O12jhw5wsDAAGazGYCqqip6enqC/r2Ghoaorq7GZrNhMpnEsFxmZiaLFy8mKioKh8MRtAHFbDZjtVrJzs6msrKSt956C5vNdkkH99+9k5KSWLNmDWvXriUpKUksQPZ4PAEtUwyLg8fGxorhuFCzZs0a8vLygL9OV5KTk1Gr1aLUoc/nw+Px0NTUxJtvvslf/vKXK+ZE+Hw+ampqrqusar709fUxOTnJwMAAGo1GdPDi4mImJjDsKC4AACAASURBVCaIi4ubV2X6lcjOzmb16tXcdtttVFRUMDQ0RENDwyWDBhqNhpiYGNasWcOWLVtYvXq1OF2dnZ3F4XDgcrkCJjUYFgc3GAykpqbOqdwIFWNjY9jtdvR6vTiS+NV4T548SWtrK3V1dezbt4/+/n6GhobCmlN9tXi9Xpqbm4mJiRELIAKxE3glJBIJjzzyCEuXLiU5ORmZTEZlZaV41/gwKSkp5OXlUVRUhFKpFH2gvr6eM2fOsGvXLt55552AZZGGxcEnJydpb28XNyZChSAIPPPMM2KCl9VqFVfx1dXVdHV1MTo6yvDwMN3d3UxNTd0Qzg3nHfzQoUMkJSWRlpYGhE4+UKVSzXHW3Nxc4uPjLzkK+wViNRoNMzMzjI+P09TUxI4dO6irq6O1tRWn0xmwCzNsDt7V1YXb7RYT3D8s8R0s3nnnHRobG8UEe5fLxfDwMO+++y5OpzOkMuKBxOv1cvr0adauXcvs7GxIe6MMDw8zMjIihn8TExNJTEy86H3+GlGfz8fk5CTDw8P09fVx4MAB3n77bVEePZCExcH9V21HRwdpaWnI5XJSUlJwOBxB3+xxuVw0NjbS2NgY1OOEGp/PR1tbG62trXR3d5ORkRGS4wqCwM9+9jOWL1/Oxo0buffee8Uq+g/T39/P1NQULpeLqqoqPvjgA6qrq2lqagraVCpsCg8qlYp//ud/Ztu2bWRnZ/OnP/2JF154gYaGhuvemLhRlBOCZadKpSIlJUUMb8Jfa1s/nAtylVyVjKBOpyMmJobExETuu+8+Kisryc/PF1t+dHZ20tLSwuHDhxkbG8PhcDAxMYHNZsNutwcke/RyCg9ha93m9Xo5ceIERUVFJCcns2zZMnHlPTo6ekNOE8LNzMwMLS0tIc/IdDqduFwuRkdHMZlMTE9P093dLTp4d3c3zc3N1NTUMD4+HtJc/7A5uMfj4ejRoxQUFGC1WqmoqGDbtm0YDAbOnDmzoBSRI1wZn8/H9PR0SHZur4lwtk+WSCSCxWIRysvLhd/97ndCS0uL8Je//EXIysoSFArFNX/e31r75BA8gtY+OVjndEG1TxYEgdHRUTweD2+++Sbt7e3ivCxSIRMhEERkBMPAjWInN4FWfUQINvTcKHbCjWNr2uVeuKYRPEKEG41rGsFvlNtUxM6AEVE6jnBTs1CnJFdNxMEj3NREHDzCTU1EZS3ApKWliX34mpqagtYd198A32QyERUVRUxMDHa7nenpaTFhbWhoCJfL9Te9IxxWBw+lpk0okEgkbNu2jY9//OMAPProo0Er+NVoNCxatIj169ezfPlyVqxYQU1NDZ2dnWJ/mVdffZWOjo6gVazfCITcwf0tDTIzM3n44Yd5/vnnF1buwnWiVCr5+te/ztatW8nKygpqA9H169ezatUqPvWpT2EymdBoNKjVahITE/F4PGKi2uTkJPv27bvpUoOvhZA7uFKpZMmSJXz84x+nsLAQi8Uitva6UUlISCAvL4/169eTmpoKQFtbW9CaiNpsNlwuFwkJCURFRYmFIh8uctiwYYPY97G/v/9vciQPuYOr1WrKysr4x3/8R0ZGRkTBzxvZwTMzM9m2bRtr1qxBIpHQ09NDXV2d2N0q0LS0tJCUlCTWll6oaXNhmdptt92GxWKhpaVFzPm5kfC3pPOLBvsvYH/1/dU0xw9bFEUikRAXF0dcXBwmkylcZgQEs9lMSUkJKpWK3t5ejh8/zp///Oeg9Vp0OBzU1dXxxBNPiC3mBEGgt7d3TqGvXq8nOTmZdevWodFogmJLsNDr9aKy9D333MMPf/hDamtrOX36NO+99x5PPPGE2Ff+owjbIvPCq3Mhym6kpaWRlZVFVlYW27dvx+l0XnIELCgoID8/H6vVytjYGIcPH+bdd98NWNuDyzE6OsqePXu47777RFGB6enpiwpFxsfHOXTo0A1xh/QrQuTl5ZGRkcHixYvR6XSYzWbi4+OxWCxiX0mn03lVIlthDxNerhl6uMnPz6eiooIVK1bwzjvvMDs7e0kHLykpITc3F5PJRFNTE8ePH+fYsWNBt8/hcHD27FlaWlowGo1iidrl3hdKOZhrQSaTERcXh0KhIC4ujpKSElavXk1+fj4FBQVotVpcLhdut5uJiQmmpqbEMrericCFzcH980W5XB52ubsPI5FI+OY3v0lxcTFarZbs7Gymp6cvWjRKJBIeeOABSkpK0Gg0/OIXv+DYsWO0tbWFxE5BEHjppZfo7+9ny5YtLFmy5KLBwt+xYKESFxfHj370IzIyMkhKSiInJ2fOHd3fHu/MmTPU1dWxd+9esS3cgnZwOP8Dmc1mFi1aFJaOUB+F2+0WpxkX9vzwo9FoSEpKIiEhAYPBgCAIKJXKkN+Nqqqq6O7u5ujRo/zkJz8hKSlpjoZnUlISDz30ED/72c9EIduFgMFgoKKiglWrVrFlyxbgr82Ldu3aRW9vL6Ojo7hcLnp6ehgeHsbhcIh6pVe7fxJyB/er3Xq9XmQyGQaD4aoWC6FCq9WSnJwsdjt1Op1iK2Q/UqmUmJgYCgsLiYmJQRAExsbGxJ3DUCKTyXC73ZdVl1AoFMTGxoali9jlkMlk5OTkUFFRQUVFhejYfimY3bt309fXJyrt+Xdor4eQO7jX68XtduN0OomKiiI6OjrsstgXYrVaeeihh8T+he3t7bS1tc3ZclcqlaSmpnL//fdjMpmYmJigvr6eQ4cOXbJdWbCQSqWsXLmS9PR00tPTyczMvGi655+Dh0pc4Er4u9Dee++9fOxjHyMlJYUXXniBZ599lnPnzgV8MRxyB3c4HHR1dVFdXc3q1atDffiPxGQysWTJEu699140Gg379u3jmWeeYXBwcM4i7ZOf/CQbNmzgtttuY3p6mldffZWf/vSnQe3geiGJiYnccsstogyjTqcT26d9mJGREfbu3Rs2WUY4fyGqVCruv/9+li5dSm5uLs8//zznzp1jdnaWffv2MTo6GpSLMCxTlNnZ2bCe8A8jlUoxGo1s2LCByspKLBYLra2t1NbWcubMGXHOp1KpyMvLo6KigpKSEvR6PceOHePMmTNXVDIIJEajkZSUFAoLC4mLi5sjVPth9Ho92dnZtLe3hy2SEh0dTXFxMZs3byYhIQGJREJzczMul4uZmRkGBwfxer1ByUtaEGHCcCKTycRIyYMPPsiyZcvQ6XTU1tZSV1dHT0+PeOL1ej233XYbq1evJiMjA7fbzd69e6mrqwupzVFRUZhMJnQ63RX3EUwmExs2bODgwYNB21m9EmazmbvuuostW7YwODjIBx98QGNjIw6HI+gNnsLu4P5NinAgk8koKyujoqKCH/zgB2g0GmQyGYIgUFBQgFQqxWKx8OyzzyKXy0lLS2Pr1q3Ex8czODjI+++/zwsvvBAQubtr4fjx44yMjDA7O8s3vvENzGbzZfsBqlQqzGZzWBeZaWlp/NM//RMKhYIjR47w1ltv4XQ6Q9K9LOwOnpSURG5ubshTZ2UyGUuXLuWuu+5i3bp16HQ6JBKJGJGwWq2ioFRUVBRarRaLxUJWVhYSiYSWlhaee+45xsbGQt5mzuv1MjAwwF/+8hcxCqVSqYDz062oqCi2bNkizsnDvZE2PT1Nf38/VqtVzNvp6Oigq6uL8fHxoB47LA7u8XjERPzY2FiSkpJCenyFQoHBYGDVqlVUVlayfPlyPB4PNpuN8fFxBgYGKCgoIDU1lfT0dGJiYsTWwEajEbvdjsPhoLPzfMmiTCYLuZM7HA5Onz6NUqkkJiYGtVoNIN51SktLsVgs4gLP3546FItguVyOTCYTF42Tk5OcOnVKXOusX7+eo0ePiqrYQU1rCEfrtpKSEuG73/2u4HK5hKmpKeHEiRPC/6/aDkj7riu9LysrS/jsZz8rTE5OCjMzM4LT6RQGBgaEr3/968Lq1auF0tJS4eWXXxYaGxsFj8cjeDwewev1Cl6vV/x/u90udHZ2CpWVlUJcXFxQ7Lyeh0QiEeLi4oTvf//7QmtrqzA1NSU0NzcL5eXlgtFovNbPu67WbYsWLRKKi4sFpVIpSCQSQaFQCLGxscIjjzwi7Nq1S/B6vcLExITw2GOPCRkZGQH53guqdZu/6bwgCOIIYzabLytcFGjS09O59dZbGR8f59SpU1RXV7Nv3z7a29ux2+14vV6effZZ1q9fz5133klWVtZFc1in00ltbS1dXV1MTEwE3earRSqVYjKZePDBB0lMTBRTSy8Uig0GKpUKk8nEt7/9bZKTk/H5fPzmN7/h5MmT2Gw2Jicn2b17NxaLhY997GNoNBpRvjuYhHWKAogZhTqdLmSOIpVKkUqlHDp0iCNHjnDq1CkOHjw45z0dHR0MDw+LU4/e3l6GhobESMTIyAjHjx9nYmIi6JmD14LZbCYnJ4eMjAwkEgmjo6N0d3cHfSqQnJwsNsH3er309PSIGkH+0LA/g1QQBCYmJkRFjWASFgf3b9X7E678u1uhWgwNDAywf/9+3n77bYaHhy8ZPtPr9ZjNZlJSUpBKpfzlL3/hrbfeorm5GZ/Ph9vtFvMjgs21LMDLy8u54447xNDhyMgI77zzDj09PUENE1ZWVvKTn/yEqKgodu7cydtvv82+ffvm/MabN2+muLgYr9dLbW0tjY2N9Pb2Bs0mIDxzcLlcLlgsFqG2tlYYHx8XhoeHhccff1zIysoKyRxcqVQK0dHRglwuv2juL5PJBIPBIHz7298W3nnnHcHtdgudnZ3C5z//eUGn0wlKpVJQKBSX/NtA26lSqYSEhAThF7/4hfDVr35VWL9+/RXn3k888YTQ398v+Dl06JBQUFAgKJXK67H1qufgFRUVwpNPPimMj48LNptN6O/vF5qamoSmpiahtrZWePfdd4W+vj5hcHBQaG5uFu68804hJSUlIGuvC8/pgpiDezwe7HY7bW1tGAwGZDIZp0+fDlkeh9vtvuyt0WAwUFBQQHl5OampqTgcDl599VXOnj0b8o2SpKQkli9fTkVFBVarVcw59+8AOp1OTCYTWq0WvV7PqlWrKCwsFCukWlpaqK+vZ2hoKOhRnu7ubvbu3UteXh5ms5no6GiSkpLE0GtMTAwzMzO0tbVx8uRJ6uvrsdlsQQ8Nhy0O7vP5aGpqwmq1olAoeO2118KelO8vo9u0aRNr1qxBqVTS29vLU089FdKteD/p6els3bqVgoICkpOTSUxM5OTJkwwPDzM2NkZfXx9ZWVlYLBaSk5P5/Oc/T3p6uiisWlVVxeHDh5mYmAh6eLC7u5vBwUGSk5NJT08nJSWF5cuXA3/dra6rq+P999/njTfeoLOz829jowcQVW7D3fQ+Li6OwsJCPvnJT6LX6zl9+jS7du2ip6cnLCVfDQ0NPP/886SlpZGTk8OKFSt45ZVXGBgYYGRkhK6uLsrKyoiOjkatVqNUKsURs729nd///vccOnQoZJmEbrebZ555BqlUKgYPLsTn8+H1esUa0lAQdgeXy+UYDAaKi4tpbm7GbreHzZZ77rmH9evXYzabOXv2LPv27ePNN98M251lfHycc+fO0dzcTE9PD9HR0dx+++0kJCQQHR2NxWIhPj5+TkGG2+2mp6eHH/zgB9TV1YU8qW0hRZQgzA6u1+tRq9VoNBqsVitdXV1hdfCioiKWLl2KwWCgoaGBkydPcurUqbDZMzU1RX9/v9iCQqvVsmzZMrGO1Wg0olarxSmAz+djaGiIuro6tm/fflN1DbtewubgcrmczZs3YzabGRgYoKWlJWiNcq6WgwcPotVqycjI4PDhwzQ1NYXVHjg/Ij/55JPI5XLi4uLE/BKDwUBWVhZ33HEHer0eOL99/+yzz/Lyyy9HnPv/EzaNHq1Wy4svvkhKSgqTk5N8/vOfp7+/f1631Pk2lk9ISMBkMpGYmMi5c+cYHx8PSmTneuz0Fw/7JbLlcjkajYb4+HjkcrlYCtjZ2cng4GCgIj5/cxo9AcPj8XDgwAEsFosY8gq3+OvAwAADAwPU19eH1Y5LIQgCbrdbTPCKcHVEVNbCwI1iJzfBCL7wOu5EiBBAIg4e4aYm4uARbmoiQrCh50axE24cW9Mu90JECDbCTU1ECDYM3Ch2chMIwYY9F+VGQ6FQiPkfAD09PTeccsI1sFCnJFdNxMGvAYVCQVpaGv/1X/9FaWkpEomEdevWRTZfFjARB79K1q5dy4oVK1i7di1FRUVIpVIaGhpu5tH7piDi4B+Bv+J/yZIlrF27ltWrV4vZfF1dXRw5cmRB9ViMcAkCXZMpkUiu+nE1n3ctj0DXjmq1WiEjI0PYu3ev0NfXJ/ZFaW5uFp5++mnBZDIJUqk0rHYG83xynX1RwvEISk2mSqUiISGB5cuX43a7MZlMJCQkoNVqKSkpISEhgdOnT4td+S/E5/MxNjZGQ0MDbW1tCyI19UKSk5NZuXIld9xxB7fccgtarRabzcYzzzxDU1MTzc3NjI+Ph7UKSSKR8Nhjj1FcXExSUhIPPPAAg4ODkbvKBczLwfPy8igqKuJjH/sYHo8HvV6P0WhEpVKRmppKTEwMWq32kgplgiBgt9spKCigt7eXhoYGsfdIX19fyHptXw6dTkdaWhqlpaVotVpkMhmzs7N0dHRQV1dHd3d32J1boVCwZMkSioqKiIuLIysri+np6YiDX8C8HLy8vJxbb72Vbdu2zXn+wltETk7OnNd8Pp/YJ8NfiTI7O0tbWxsHDhygurpalMEOlwP5q2VSU1PJz88HEFUpRkZGaG9vZ2hoKCy2+fE7eHJyMvHx8ajVavLy8ujr6wt5t9vrxS/wCn9txuTvkisIAj6fb9557fNy8OXLl7Ns2bI5zwmCwPj4OH19fYyNjV30WmdnJ1qtFrPZTFRUFMnJycTGxrJ48WKysrIoLi6ms7OT/v7+sNT3abVarFYrP/7xj0XnBnjhhRd47bXXeOedd8Ketw7nB4qpqSm6urpISEgQz6O/CeeNwJo1aygqKhIv1MzMTNauXQuclytvb29n69at86r0mpeDv/nmmzQ3N5OcnMzIyIjYrcrhcDA8PDxH18bP5OQkCoUCrVaLwWDg3nvvZe3ataLmuslkoqSkhGPHjoWlhC0mJoaVK1eSmpqKx+OhpqaG119/naNHj1JfX7+gwoKCIHD06FFR8cFisQS91998SEtLw2g0EhMTQ3p6OhUVFeTm5iKTydDr9WIXXzhfsaRQKPjud7/Lzp07OXHixHUdc14OfvDgQerr67FYLPT09Igj7vT09BWVsSQSCWazmbKyMsrLy/9qkFyO0WgMS0/r+Ph4srOzKS8vJzo6mv7+fo4dO8Zvf/tbRkZGFuTc9syZM+Tn5yOVSklOTiY6Ojos7ZwvRCqViju+crlc/C2Li4tJSEjAarWKArpWq1XsbQ6ItaQqlYq4uDjuv/9+6uvrw+Pgftm3c+fOXfuB5XIqKyvJz88XVdZmZ2fp6enhlVdeCalamZ/HHnuMjRs3kpmZydDQEHv37uXXv/41vb29Ye/ZcjnOnTtHX1+fWMT9wQcfUFVVxfDwcNhs0uv1ZGVl8bOf/Qyr1YpGowHOF0qrVCpUKpW4l9Da2jpnKnghUqkUjUZzWfWKqyHkGz0ymYzi4mI2bNjAJz7xCTIzM8XXnnrqKd577z2amppCOv9Wq9WUlJRQVFREWloacrmc48ePc+7cOWZmZhZ0hbrfNv+iMysri5KSEt59992Q2yKVSiksLGTLli2sWrWKoqIiVCqVOIL7R3OXy8Wjjz7KwMAA09PTJCQkEBUVRXZ2Nvfeey8mk0ls1t/d3T2vViIhc3C/qoLVamXFihVs3ryZoqIilEolMzMztLS0cOjQIU6dOnXJuXuwkEqlaLVaFi9eTHx8PDqdDkEQxO5Rl5qWmM1mtFqteGu12Wxhj6rAX+VLwiWs65eFWblyJeXl5XO0l3w+H+3t7aJa8Z49e7DZbHg8HrEfZGxsrPh+v/rDsWPH5hUVCpmDR0VFiZo4y5cvF/vW+Xw+RkZG+NWvfsWJEydC7igqlUpULfYvcARBYGpqivHx8UvKX69atUq8IOD8WuT1118P60jvP7ZUKg2b4JRCoeDuu++mpKQEo9EoPu/z+ZiZmeHFF1/k7NmztLS00NbWJk77JiYmuOeeeygtLRUvTrvdTnd3N08++eS8fCIkDr5o0SLWrVvHD37wA6KiouaEsvwKBAMDA8zMzIR8ruvz+cQNE4PBgFQqRRAETp8+PSdLsLy8XFQ/3rZtGzk5OZjNZuC8MOz3vvc9tm3b9je7k5iYmEhhYSFLly6dMxKPjY1RW1vL008/zcGDB7Hb7ZfsQ+mPrvjp7u5m3759jIyMzKu3YkgcPD09ndzcXJKTky/SdJRIJBgMBm677TasVit9fX309vbS09OD3W4PesvimJgYrFYr6enpaDQaPB4PDoeD8fFxXC4XEokEk8lEeno6JSUlJCUlkZeXR3x8vLhLazAYiImJ4Y477uDgwYM0NzeHTZMyXCQmJlJeXk5UVBQKhQKv18vQ0BAnTpygqqqK48ePMzw8PCfMKpfLUalUJCUlkZ6ejsViAc4HL9rb22loaJh3o86QOHhGRgaLFi3C6/VeJFzqj30//PDDtLa20tbWRlVVFe+//z4dHR309PSISsPBwGKxiJtMcF4/aGhoiImJCdxuN3K5nPT0dHJycigsLGTNmjWiwsPY2BhDQ0MYjUYSEhJ46KGHmJmZmSN1EipCLcP4YaxWKytXrkSpVIpTvPr6ejGG3dHRcdHf+DtzrVmzhiVLlpCYmIjP56O5uZm6ujrq6urmfUcPiYMfO3YMr9eL0WgkPz+fmJgYdDrdRe/zXwhr167lm9/8JjU1NezZs4f/+7//Y2xsLOAtjHU6HWazWWyHNj4+Tnt7O++++y6NjY14PB5ycnL44he/SEVFBVlZWdhsNmpqaqipqWHnzp1otVq2bdvGl770JQwGw7xCWvMlnKrRjY2NbN++ndjYWLxeL11dXfzrv/4rw8PDl52yrVixgo0bN/LVr34VlUqFy+WisbGR73znOzQ1NWGz2eYdzw/Jr9Hb2ytq2mRkZBAVFYVer0ev15OSkkJKSgpLly4Ve0rLZDKUSiWLFy9GpVLh8/nYtWsXJ0+eDJhNUqmU9PR01q9fz+bNm4HzmybHjh1jx44dCILAkiVL2LBhAxUVFVgsFpxOJ6dOnWLHjh00NzfjcDi46667WLJkCS6Xi+bmZvr7+4MubroQGRoa4tChQ2IEzG63MzQ0xPT09EWjsFQqJTc3Vzz3Go0GQRAYHR1l//79dHd3MzExEZDNqpA4uM1mw2az0dTUJIbidDodRqORJUuWsGTJElH2wi9aCucb0vt3NTs7O2ltbQ2Y80ilUhYtWkRJSQmlpaXAeQc/dOgQJ06cID8/n9LSUjZt2kROTg6jo6N0dXVx+PBhDh8+jN1up7CwkLVr12KxWBgaGqKmpoaenp6wpBj4pycejycsOTyTk5NMTk7S0tJyxffKZDJWrFjBihUrKCwsBM4vRtva2jh8+DA2my1w3yEcIlQffkRFRQmf+tSnhFdeeUVobGwUfD7fRY8dO3YIX/jCFwJWSKBUKoVHHnlEOHTokCjuumnTJrGIYPv27UJLS4v42hNPPCEsX75ckEgkQm5urvCNb3xDmJ2dFTwej3Dy5Enhxz/+saBSqa4pOT8Q5y4hIUH493//d/E8PfHEE0JZWVmgfpugFDxERUUJ4+Pj4rn1eDzCk08+KWzZsmVhFTwECqfTyd69ezl79iy33HILX//611m8eLFYuQ7nox3+VXYgkEgk6HQ6FAoFLpeLmpoaxsfH0Wg0JCUlYbVaiY6OZnZ2lj/84Q/s3r2blpYWrFYrn/vc56isrEQikfDHP/6R/fv3s3fv3rBrDMH5mHI4t+mvRG5uLhs3bhR3OP13nra2Ntrb2wN+vAXh4F6vl4GBAYaGhpDL5XR2dpKRkTHHwf35wYHEv3Xs8/kYHh5mdnYWuVyOXq8XcyD8u5pGo5HS0lISExMpKipCq9Wyf/9+3nvvPU6ePElra2tAbbte3G53WPSErgaJREJCQgLFxcVzJFfa2tro7OxkZGQk4MdcEA7ux+fzYbfbaWlpYfXq1XNeGx8fD2kiv/8WJ5FIWLRoERs3biQlJQWVSsXAwACHDx/mhz/8IQ6HY8E61ELDnymakZEBnD/HNpuNp556ipqamkvuGs/7mAH/xP+PXyn49ttvp7+/n87OziumPPo3fXJzc8WFJpwf4bu7u68ra/FyeDwe3n//fUpKSiguLmbjxo28/fbbREdHk5mZSUxMjLjgvfPOO1EoFEilUux2O6+//jr79+9ncHAwKHeW+eDPo15I+LsTbNq0iS1btrBixQpkMhlnzpyhqqqKN954g+Hh4aDE8YPi4BqNhmXLllFaWsq6det48cUXPzL9VSKRoFaryc7OpqysjMzMTHF64r/Ke3t76e7uDqid/px1n8+HTqfj1ltvpaioiPj4eGJiYsSYtl6vFytompubOX36NM3NzQuisufDxMbGkpycTE9PT7hNAc5fcPHx8Sxbtow77riDpUuXolQqOXz4MAcPHqSqqoqRkZGgrV8C7uASiYT4+Hhuu+027r77brKysnj22Wcve8L927UJCQls3LiRyspKsrOzxZ05n89HV1cXbW1tAXVwQRCYmZnB4XAwOTlJVFQUd99995wdwQv1O10uF6Ojoxw5coSamhq6uroCZst88J8jn8+HVColMTGRrKwsjh49Gm7TAFAqlWRkZPDQQw+xadMmdDodU1NT/OlPfxIDC8HcgQ24g6vVah577DHWrFkj5nrHx8djsVjweDxz8qvVajXr169n3bp1fPrTnxYT4v07ch6PB5vNxo9+9KOAy/n5fD7q6+t56qmnOHr0KA8++CCFhYVzEn5aWlqoqqri3LlzdHV10d3dzbFjxxaUNptOtAAAAu1JREFUFqTNZqOnp4fW1tY5ufULhdjYWHJycti6dStyuZzJyUk6Ozv53e9+x+Tk5I0n5e3z+ejo6BB3JgEeeOABli9fTkdHB06nU6ys1+l0FBUVkZubS2xsLHK5XBxBZ2dnqa2t5d133w3aAsRvq8PhYGhoSCza9Z90m83G4OAgNpsNu92Ow+EImWrw1eLxeHC73aJdCQkJC8LRZTIZcXFxlJeXU1pailwuZ3R0lGPHjvHnP/9Z9INgExQHb21tFaXsdDodlZWV3HLLLXR1dc1xcH8F+4W5w7OzszgcDkZGRjh27BivvfZaUHuQjI2NMTY2tuAaD10t/sQvf3KXyWTCarWG2arzueGZmZmsWLGCoqIi4HzKxvHjx3njjTdCVrwdcAf3eDwcO3ZMTGDaunUrcD6xyZ9PfTn88+3XXntN3Fi5VBZahLn4L9DS0lKUSqWofhzO7MKYmBi+8pWvsG7dOhITExEEgZ07d7Jv376QhnsD7uCCIDA4OMjOnTuprq6mpqaG7OxsFAoFo6OjlJSUoNFo8Pl89PX1idUeDQ0NYler6upqxsbGwq58fKPQ1NTE9u3bKSoqoq6ujuPHj4fVuU0mE8nJyej1ehQKBcPDw1RVVVFVVRXy6E5QwoTT09N0dXUxOjqKRCKhra0NpVLJ6Ogog4ODcxzcH81oaGhgeHgYm81Gf39/MMy6aRkbG6Ouro7du3fT1tZGY2NjWO0xm81kZ2eTkJAA/DXTsL29PfSZlgsh2SpQj4idAX9cV7LVtm3bhKefflrweDxCb2+v8Mc//lHIyckRFApF0M/phx8RGcEIAaelpUXM3X/ppZf47W9/S3t7e1jCqwsqFyXCzcHQ0BDV1dX86le/Ys+ePTQ2NoZt7yCiVR8GbhQ7uQm06iNCsKHnRrETbhxb0y73QkQINsJNTWSRGeGmJuLgEW5qIg4e4aYm4uARbmoiDh7hpibi4BFuaiIOHuGmJuLgEf7fhjUYTeCjYFgDAIIEKi/Ey+h6AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 216x216 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "def plot_digits(im):\n",
    "  plot.figure(figsize=(3, 3))\n",
    "  grid = gridspec.GridSpec(4, 4, wspace=0.05, hspace=0.05)\n",
    "  for i, g in enumerate(grid):\n",
    "    ax = plot.subplot(g)\n",
    "    ax.set_xticks([])\n",
    "    ax.set_yticks([])\n",
    "    ax.imshow(im[i,:,:,0], cmap='gray')\n",
    "\n",
    "plot_digits(images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rVeSdnNJr0cV"
   },
   "source": [
    "Now we can create our GAN.  Like in the last tutorial, it consists of two parts:\n",
    "\n",
    "1. The generator takes random noise as its input and produces output that will hopefully resemble the training data.\n",
    "2. The discriminator takes a set of samples as input (possibly training data, possibly created by the generator), and tries to determine which are which.\n",
    "\n",
    "This time we will use a different style of GAN called a Wasserstein GAN (or WGAN for short).  In many cases, they are found to produce better results than conventional GANs.  The main difference between the two is in the discriminator (often called a \"critic\" in this context).  Instead of outputting the probability of a sample being real training data, it tries to learn how to measure the distance between the training distribution and generated distribution.  That measure can then be directly used as a loss function for training the generator.\n",
    "\n",
    "We use a very simple model.  The generator uses a dense layer to transform the input noise into a 7x7 image with eight channels.  That is followed by two convolutional layers that upsample it first to 14x14, and finally to 28x28.\n",
    "\n",
    "The discriminator does roughly the same thing in reverse.  Two convolutional layers downsample the image first to 14x14, then to 7x7.  A final dense layer produces a single number as output.  In the last tutorial we used a sigmoid activation to produce a number between 0 and 1 that could be interpreted as a probability.  Since this is a WGAN, we instead use a softplus activation.  It produces an unbounded positive number that can be interpreted as a distance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8zLMNX5Xr0cW",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "class DigitGAN(dc.models.WGAN):\n",
    "\n",
    "  def get_noise_input_shape(self):\n",
    "    return (10,)\n",
    "\n",
    "  def get_data_input_shapes(self):\n",
    "    return [(28, 28, 1)]\n",
    "\n",
    "  def create_generator(self):\n",
    "    return tf.keras.Sequential([\n",
    "        Dense(7*7*8, activation=tf.nn.relu),\n",
    "        Reshape((7, 7, 8)),\n",
    "        Conv2DTranspose(filters=16, kernel_size=5, strides=2, activation=tf.nn.relu, padding='same'),\n",
    "        Conv2DTranspose(filters=1, kernel_size=5, strides=2, activation=tf.sigmoid, padding='same')\n",
    "    ])\n",
    "\n",
    "  def create_discriminator(self):\n",
    "    return tf.keras.Sequential([\n",
    "        Conv2D(filters=32, kernel_size=5, strides=2, activation=tf.nn.leaky_relu, padding='same'),\n",
    "        Conv2D(filters=64, kernel_size=5, strides=2, activation=tf.nn.leaky_relu, padding='same'),\n",
    "        Dense(1, activation=tf.math.softplus)\n",
    "    ])\n",
    "\n",
    "gan = DigitGAN(learning_rate=ExponentialDecay(0.001, 0.9, 5000))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "69GHTt_2r0cb"
   },
   "source": [
    "Now to train it.  As in the last tutorial, we write a generator to produce data.  This time the data is coming from a dataset, which we loop over 100 times.\n",
    "\n",
    "One other difference is worth noting.  When training a conventional GAN, it is important to keep the generator and discriminator in balance thoughout training.  If either one gets too far ahead, it becomes very difficult for the other one to learn.\n",
    "\n",
    "WGANs do not have this problem.  In fact, the better the discriminator gets, the cleaner a signal it provides and the easier it becomes for the generator to learn.  We therefore specify `generator_steps=0.2` so that it will only take one step of training the generator for every five steps of training the discriminator.  This tends to produce faster training and better results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "lP7x5ZT1r0cc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Ending global_step 4999: generator average loss 0.340072, discriminator average loss -0.0234236\n",
      "Ending global_step 9999: generator average loss 0.52308, discriminator average loss -0.00702729\n",
      "Ending global_step 14999: generator average loss 0.572661, discriminator average loss -0.00635684\n",
      "Ending global_step 19999: generator average loss 0.560454, discriminator average loss -0.00534357\n",
      "Ending global_step 24999: generator average loss 0.556055, discriminator average loss -0.00620613\n",
      "Ending global_step 29999: generator average loss 0.541958, discriminator average loss -0.00734233\n",
      "Ending global_step 34999: generator average loss 0.540904, discriminator average loss -0.00736641\n",
      "Ending global_step 39999: generator average loss 0.524298, discriminator average loss -0.00650514\n",
      "Ending global_step 44999: generator average loss 0.503931, discriminator average loss -0.00563732\n",
      "Ending global_step 49999: generator average loss 0.528964, discriminator average loss -0.00590612\n",
      "Ending global_step 54999: generator average loss 0.510892, discriminator average loss -0.00562366\n",
      "Ending global_step 59999: generator average loss 0.494756, discriminator average loss -0.00533636\n",
      "TIMING: model fitting took 4197.860 s\n"
     ]
    }
   ],
   "source": [
    "def iterbatches(epochs):\n",
    "  for i in range(epochs):\n",
    "    for batch in dataset.iterbatches(batch_size=gan.batch_size):\n",
    "      yield {gan.data_inputs[0]: batch[0]}\n",
    "\n",
    "gan.fit_gan(iterbatches(100), generator_steps=0.2, checkpoint_interval=5000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "UW60zOZGr0ci"
   },
   "source": [
    "Let's generate some data and see how the results look."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "fSQtVhSer0ck"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAALgAAAC0CAYAAAAn8ea8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOx9d3hc5ZX+e++d3mc0o9GMerMlWZYlW7blhm2MAYMBE5xQAoS2gWyyyeYXSEISnJAFQtjNbmCzkLIEEocACaYlxL3bSLJs2Zas3nubGY2m9+/3h/d+kYxtZHuKTPQ+zzyAhHSP7pw593ynvC9DCMEsZvFZBZtoA2Yxi1hi1sFn8ZnGrIPP4jONWQefxWcasw4+i880BJfyPzMMM6NLLoQQBpi1M4qwEEIMwMy3lb+n52I2gs/iYuhJtAFXilkHn8VnGrMOPovPNC4pB59F4iEUCiGXy0EIgcPhwGwn+uKYdfCrAAzDwGw2w2AwwGAwQKVSIRKJwG63Y3R0FHa7HePj4/B4PIk2dcZh1sGvAnAch+uvvx4333wzli9fDplMBgDweDz48MMPUV1djcrKSrS3tyMcDs9G9UlgLuVmxLNUpFAoEIlEEAqFEAwGp/WmXS3lt+nayTAMRCIRbrzxRtx3331Yu3YtVCoVGIYBIQThcBhOpxOBQAB+vx9erxd79+7FoUOH8M4770TD1BOEkPLp2JpoXKhMOOMiuFwuR3p6OpYtWwabzYaWlhZ0dnZO28njCY7jkJaWBq/XC5vNhlAoFNXfL5FIoNfrsWrVKuTm5kKhUIAQgs7OTtjtdvh8PuTk5EClUiE5ORmEEPj9fmi1Wuh0Ovztb39Df39/VG26VGRlZSEjIwOFhYUQCARgGAYM80lf5D+wgUAAJ06cwPDwMEZGRq74PZ8RDs4wDBQKBRiGgclkwurVq/G1r30NDQ0NePvtt9HX14dwOIxwOJxoUwEAAoEAQqEQUqkUCxcuxOjoKOrq6uByuaL6IVSpVMjOzsZ1112H9PR0MAwDl8uFyspKdHR0YGJiAnfccQcyMjIgEAggFotRWlqKkpIS3HLLLRgcHMT4+DjcbnfUbLpUlJeXY926dfjiF78IuVwOlj1/4Y4QgkAgAIfDgRdffBFVVVXweDxwOp1XdE9nhINnZmZiz5490Gq1EIlEAM7+wdXV1WhsbITP55sxzg0Aa9aswdKlS3HTTTdBJBLh6NGjYBgG1dXVCAaDUbkGx3GYP38+7r33XmRlZYHjOIyMjOCdd97BG2+8ge7ubjAMA4fDgfnz56O0tBTl5eWQyWQQCARQqVT43e9+hz179uDBBx+E1+uN+xOQ4zjceuutuO6662gAuxAYhoFYLIZer8dTTz0Ft9uN4eFh3HjjjRgdHYXf778sGxLm4GKxGFu2bIFSqYRWq4XZbIZYLAYABAIBfPTRR6isrMTIyAgikUiizKRgWRZGoxFPPvkksrKykJSUhKSkJNTX12N4eBiEEOj1ejgcDni9XkQiEZorXw74VK2kpAQcx2F4eBhNTU3Ytm0benp64HA4wDAMjhw5gubmZhw4cAAZGRm48cYbUVFRgaSkJKhUKixatAjPPvssnnvuOYyNjUX5rlwYAoEAycnJ0Ov1UCqVAID+/n4QQiAQCKDVaiEUCsFx3JSf4x2d4ziEw2GYzWa43e6ry8HVajWysrLw8MMPQ6fTQSgU0u95PB709/dj7969OHnyJCYmJmISefgDHCGEHmb5rzMMA41GA5lMBolEApZlwbIscnNz8dhjj4HjODidTrS3t6OhoQHd3d3wer1QqVTw+/3w+XxTrnM59ovFYmg0GqSnp4MQgq6uLtTU1ODYsWNTziPt7e1ob28HABq9ZTIZCgoKkJSUhMzMTDzwwAP461//ioaGBthsNgQCgSjcwYtDIBAgLS0NSqUSIpEIwWAQra2ttJSZmpoKsVgMsViMpKQkyOVymqPzAY3jOKjVahr4LguEkGm/AJBovL785S8Tu91OzkUkEiHV1dVk9erVRKlUEo7jLun3TtdOlmWJWCwmpaWlZO7cuSQ1NZVwHEc4jiNisZhotVry5JNPkh07dhCv10uCwSAJBAIkEAiQcDhMgsEgOX36NHn00UfJnDlziMlkIsnJySQrK4uoVKqo2KlUKsnXvvY1Mjw8TJqamshjjz1GkpOTP/V3MwxDNBoNefjhh0l9fT0Jh8MkFAoRp9NJXn31VVJQUEBYlp3uPT1+ue+9TqcjTz75JKmvryd2u50MDg6SDRs2kIyMDCIUColIJCISiYRkZGSQF198kbS1tRGfz0eCwSBxu93EYrGQ5uZmsmjRIqLVaqd9T899xTWCMwyDp556CmvXroVCoaBfD4VCGBsbw9atW1FVVYX29nYEAoGYRG4+t123bh1uuOEGCIVChEIhjI6OIhAIQCQSwWg0IiMjAzqdjkaPYDAIv9+P8fFxWCwWdHd3QyQSgWEYBINBeL1eWrKLBrxeL06dOoXXXnsNOTk5GBwcnNZhkRACl8uFXbt2YePGjUhJSYFOp4NAIEAgEIDNZotLLh4MBtHR0YHOzk6Ew2FaxgwEAlOelg6HA8eOHcOqVavAMAxYloVIJEIkEoFUKoVAILjgwXQ6iHuKsmLFChQVFYHjOIRCIfh8PjgcDhw6dAj79u1DY2MjHA5HzBoWQqEQmZmZWLVqFcrLyyEUCkEIwdDQECKRCEQiEZKSkjA2Noaenh7U1dUhEAggGAzSU34wGITP54PBYIBarYbL5YLT6bzsPPF8CIfD6O3txaFDh8AwDNxu97Tf6FAohMHBQVgsFrhcLuh0OrAse0VngksFXxXhP/herxc+nw+hUIjawDuzyWSiqeDkv8HhcMDtdl/RwT2uDs6yLLKzs2nN1uVyYWRkBI2NjXjiiScwMTGBUChE87BYvBkymQxZWVlYtmwZWJalzmqxWKBSqQAAfX19ePfdd3Hs2DHU1NTQqMeyLJRKJcrKylBcXIxrr70WTU1NNLJHE4QQ9PX1YWxsDAaDAX6/HxqNBk6nc1o/Hw6HYbPZMD4+jszMTLAsC4lEApVKBavVGnNH5zgOWq2WPgFdLhd8Pt8UZ5VIJEhPT8f/+3//D1qtFizLghCCYDCIkZERnDlzBt3d3VdU5oybg/OPH5fLhbq6Opw8eRL//u//Dr/fj2AwiNHRUerU/MEvFjZce+21WLhwIZRKJfx+Pzo6OtDU1IQ333wTgUAALpcLw8PDcDqd8Pl88Pv9tEQZiUTgdDrhdDoRCoVgMBiwfPlysCyLsbGxKf9vNMBHwePHj0Or1SIjI4NWIqaDn/3sZ6ivr8err74KoVCI5cuXY8uWLXj66acxNjYGt9sdswqV2+3Gzp07MTw8DKPRCLlcDrvdDkIIPUA/+OCDuPXWW6HX62k1JRQKoampCTt27MAf//hHeDyeq6sOfujQIbjdbhw/fhxtbW1TymnnHGhjgrS0NOh0Oto1czqdmJiYgEgkgtVqxejoKHp6LjznzzAM/H4/nE4nxsbGoNPpUFBQgFAohLa2NvT19cFqtUbNcXgnFwqFEIvFl5RmjI2Nob+/H/39/UhPT4dOp8O8efOwdOlSnDhxAl1dXTFzcP4J0tLSgtHRUej1eixevBhSqRQSiQQ6nQ7XXnst8vPzwXEcCCHwer0YGxvD0aNHUVtbi76+viu2L64RnBCCP//5z7BYLGhpaaFfZxiG5sKRSCSmTR2NRgORSASPx0PzQgAoLCxEOBw+7+OQ47gpByC/3w+73Y7Ozk7k5uaitLQUc+fORWVlJQ4ePIiJiYmoluJYloVUKoVMJrtos+RcEELgdDrR0tKC5ORkKBQKZGZmYt26dbBYLOjr64taY+p81/b5fOju7sbo6ChcLhe2bNmCuXPnwmw2Q6PRTDlABoNBOBwOdHR0YPv27WhpaYnKdGTcHDwSiSASiaCqqmpKBGJZFmKxGCkpKRCJRPB6vRgYGIj6XAdw9qb/+te/Rm1tLa677jqUlZXBYDDAbDZDJBKhq6uLtsEDgQA4jqPtb74T19PTQ0/5wWAQ+fn50Ol0YBgGnZ2dtFoRTTAMg9TUVBgMhktycOBs7tvc3IyKigrIZDLI5XJ6wI/mofhCiEQiYFkWCoUCS5cupWMFk5/afIo6PDyM/v5+Ol4QDR+Ie4pyvujMsiwMBgOuvfZaeDwevPnmm7BarTGJ5OPj4zhz5gyCwSDmz58PiUQChUIBlUoFuVyOpKQkmEwmeDweGrFVKhUaGhrQ3NwMp9MJmUxGO5cKhQJCoRButxv79u1DX19fVO3lOA5z585FRkYG5HI5/XBN99GtVquxcOFCiMViWq6Ld2c4EAjAYrEA+PvTkEckEplSXeE4DkqlklZVrpoU5UIghEAoFCInJwdr166F1WrFu+++e8mRarrwer3o7++H2+3G2NgYBIKzt4BlWQiFQuTm5mLevHnwer30ZwYHB9Hb2wuv10tb5ADo7Izb7UZ/fz9qa2tht9ujai8fvZOTkyEWi2E2mzE4ODitx7dYLIZWq0VBQQHtJvp8PkxMTMQlevMIhUKfGJqafN7iS7Aejwfj4+P0XBYNJNzBgbOO8u1vfxt6vR6VlZUYGhqKSYrCIxAIwGq14uWXX0ZycjLkcjm8Xi8MBgPy8vJwww03QKVSYWJiAr29vXj99ddRV1eHnp4eeL1eSKVSAGfnRdxuN6qqqvD222/DbrfHxG6fzweBQID09HQ8/vjjePnll9HU1DSlpnw+5Ofno7CwECqVipZEx8fH8cEHH6C9vT1ukZzjOEilUlqLB0CvzTv5wMAAKisr8eqrr2JoaChqjb6EO7hCoUBycjIyMjI+UZaLJSKRCDo7O9Hd3Q3grBOJRCLIZDK8//77kEgktGzY29tLGzlisRhLlizBsmXLkJaWhjNnzqCqqgrHjh2Lid3BYBA7d+5EXV0dkpOTkZWVBY1Gg8zMTIyMjNDBLj7XBf4+Z7NixQosW7YMHMchEolgfHwc7e3taGlpifqT5mLgHZx3bn72m2EYhMNhOBwOtLa2orW1FVarNaqz/wl38OzsbJSVlUGpVKK/v59O5sUafIUhGAzSRyTfzOnq6oJQKEQ4HEYoFKJRWSQSITs7G+Xl5SgpKYFGo8HQ0BBtyMTCbkIInevu6+uDx+OhlZ7zXY+vSGVkZKC4uBi5ubm0ccaXNx0OR1wGribbNLnEOTmK+3w+tLS0oL6+Hm1tbfQDGy0k3MFvuukm3H///RAIBKiurkZ1dXXcrh0KhWgbnncW3hHOzVH5pYyNGzdi06ZNyM7OBgAMDw/DZrPF/Knj9XppnZg/N5ybokQiEXpIu+GGG1BRUYH8/HwaKflIH++5cL5SNhkMw9DD55///Gfs2rULg4ODUb/2ZTs4n0+xLAuBQIBgMHhJ8yMsy6KoqAiFhYXIyMhAJBLBzp07sWfPnss16ZLAjwrw/34uBAIBrcsTQqBWq5Gfn4/HHnsMer0eQqEQPp8PVVVVaGtri+uMx4Vq1wzDYN68eVi0aBEefvhhpKenQyKRIBKJwGKx4OjRo/j9738f1+gNnF1o+epXvwqtVkujN8MwOHToEPbs2YN33303ZltHVxTB+TlpvrzD51afBr5Ve8cdd2Du3LmIRCI4c+YMhoaGpj1rEQ1cyCkZhoFWq6W5q9vtRllZGVauXAmj0UjTl0AggLGxMTgcjrjZfCFwHIeUlBQsXboUa9asgdlshkQioUNL+/btw9GjR+N6uATOBgqdTkdLlTwcDgfq6+tRWVkZ0/t3yQ4+uXzDH2rC4TDtRH6ag7MsC7Vajby8PDzyyCOQyWQYGxvD3r17YbFYEr69wz+VkpOT6bjm8PAwrr32WmzatAkSiQTA2UMpPzqbyJ1H4KwTSaVSFBcXY/369Vi/fj210+VyoaurC2+++SYaGhpikgZcDGq1GqmpqZg3b96U1Kq3txcnTpxAVVVVTK9/SQ7Od/b4R1woFKJfEwqFdDb6Yo/roqIi3HbbbXjggQcgEAjw0ksvYdu2bWhra4tZ2/hSwFcgxGIx5s6di9LSUqSkpGDRokXIy8ujf2N9fT22bNkSlc3vKwHLstSx77//fphMJshkMjrjfvz4cfz85z/HqVOnEkIM9L3vfQ/XX3893dryeDwYHBzEjTfeCJvNFvPrX5KDRyIRBAKBKVE2HA5Tx7xY9ObHNR955BEsWbIEGo0Gb775Jj7++GP09vbGtfFwMfA5bl9fH10QKC0thUajAcMwGB8fx+7du3Hq1CmcOXMm7vnsueAPv/wKGH9IDgQCOHz4MI4ePYq2tjb4fL6EfBDNZjNMJhP979OnT2Pbtm0YHx+PS0C7JAcnhHyikcFHNAAXPKFzHAeFQoH09HSsW7cOOp0OfX192L59O11wmCng/0aLxQKn04mRkREAZ2eXbTYbBgYGsHXrVroHORPATxy6XC66ZxoIBHDy5EmcOHEiZiXM6UChUNClY0II2tvb8cEHH8S0kTcZV1wm5DtR54vA/KRgcnIyli5diieeeAKhUAivvvoqfvGLXyT0xn8a+Gk4n8+H7du3Y8eOHfTrMwnhcBjHjx9HZ2cnxGIxSkpKYDab4fF40N3djYGBgYTazK/6CQQChEIhjI+Po7e3N25nrZjUwRmGAcdxSEpKwv3334+ysjIoFAq0tbXhrbfeQltbG8bHx2ecs1wMM9nWYDAIu92OrVu3oqioCKmpqRCJRGhtbY36ptGl4sUXX0R1dTU2b96M1tZW1NTUxJXjJi6NnkAggNHRUZw4cQKVlZWYmJiIx2X/YcDn3a2trXC5XEhOToZEIqGL1IlEXV0dvF4vNBoNWltb0dzcHNdKWUzJN/mKBF9PnswXEguQzxj55gzAFZNvsiwLjuMgFArpqG6MRhriT77JH3Zm8Y8Lfjzg0yYfY4WYpygzOXedRfyQKD+Y1eiZxWcalxrBLZi50nKZk/591s7o4GqxNfNC35ixCg+TwS+pArjoyOfVcni7WuzEZ0AINuHz4J8GlmUxZ84cunbV1NQEl8uVsEPLPxhmasSeNq4KB09PT4dWq0U4HEZHR0dcOfZmcXVjxju4SCTCvffeC6fTicHBQSoXMlt+nMV0MKOrKDKZDMnJyQDObrArlUqMjY3NOvcspo0ZG8EZhoFUKoVer4darab8fA6HI26TaJ9mm8FggFarhdPpxOjoaFy3kWYxPcxYBxcIBFCr1UhPT0dBQQEGBwcxMDBAKZbjjcmbTCKRCGlpabj55puxcuVK1NXV4YMPPkBdXV3CN5KuRky+t9E+W824MiFPl7ZgwQLceOONuPvuu6HRaPBf//VfePvtty86ahmr8ptarcY///M/Y+PGjXRxQyqVUv5rXlfouuuum5bsXazs5BfAeX4Xk8kEhUJB/3tgYAAtLS2XsvwQMyFYiUSCoqIiLFiwAI888ggGBwdRU1ODn//855eVgl4VZUL+DRKLxSgrK0NJSQlSU1PR3NyM3t5eDA8PxzVCchwHuVyOL33pS1i3bh3mzZs3pQbPV3OkUilMJhO0Wi0CgUBMhsp4vhOj0UgH2FiWhUwmg9FoRFZWFgQCAX2JRCL6AeQ4DoFAAKdPn4bdbqfkpvGuRPFj1EKhEBqNBvn5+Vi4cCHmzZsHs9kMpVKJ8fFxvP3221FbgplRDs7/8WKxGEuXLsWcOXMgFApx6tQpDAwMxHwacTIEAgEUCgXMZjOeeuopqNVqKsQ6Pj5O1QokEgnkcjkAwGQyUSWDaIPfiiorK4NKpYJYLIZIJILBYMDixYuxbt06SCSSTzzu+Q2l3t5eCIVCtLa2UrGBRGiPCoVCyOVy6HQ6ZGdnY+7cuVCpVFCr1UhJScHcuXOxe/fuz6aD80xSkUiEcke7XC689tpraG1tjYsNfJRZt24drrvuOtx2221UDcJiseDtt9/Ge++9h4GBAUgkEnzpS19CeXk5CgsLsXbtWvh8Plit1qjbZTKZsGDBArzyyitUX1IoFNKIPVmKcTLC4TClpOZlDxMhCguAktz7fD7YbDYMDw+jsrISq1atogWEM2fORHU/d0Y5+GRxIp1OB7lcjnA4jIGBAUrSE2vwj8+srCzIZDIMDg6ira0NnZ2d6OzspPzh/P6jTqdDUlIStFotNmzYgMbGRpw6dSrqdvGb81qtdorECy+KRQihu48AqCRIOByGx+PBiRMnqE5loptkk5ll+XlxPrDwFG/RwoxycIZhIJFIYDKZoNfrIZVKEQqFYLfb47Z1z9OMMQyDsbEx1NbWYnBwEHV1dWhtbUV/fz8ljgRA1Xz59MFgMETdJpFIhIKCAixZsmQKDbLFYkFPTw/l19br9dR5RSIRneEJhUI4efIkurq6ZkwPgd/V5QWy+P3dc/nDrxQJd3D+D+OrE8XFxfjWt76FgoICCIVCjIyMxExS8Hzw+Xzo6+vDb37zG2qbQqGA1+uF3++ndvCMqXl5eUhJSaEEl/zfEq3DMMuyyMjIQElJCUpKSujBtr29HT//+c+xfft2OByO86pm8B8+s9mMw4cPz6g6vVAoxMMPP4wHH3yQPm18Ph8GBwejWgZOqIPzUnO8KCwvklRSUgKWZdHd3Y2qqqqE1L15B2UYBh6P5xMHMqVSiXnz5kGlUlFuxv3790dFOIm/rk6nQ3p6Oj73uc9RbVEeNpsNR48ePa9SGi+vwuuQ2u12BIPBGVGj55t3P/rRj1BRUUHTqkgkAqvVioMHD04RH7hSJNTB+UpFUlISpFIpVVdISkqiEn9Hjx5NyGn/XDUCADQyMwwDlUqF4uJiSCQSSjFRWVkZNWo0oVAIrVaL3NxclJSUQK/XT+H/5jgOIpEIEomEOr5YLKYqZhqNBklJSVQzKNbqddOBUqlEbm4uFi1ahGuvvZaSmBJC4Ha7qTZmNNOohDk4wzCQyWTQaDQ0F9uwYQPmzJkDkUiEzs5OfPzxx3jjjTcSRunGpxsymYxKUPOHIpPJhA0bNtDvjY+P45133sHQ0FBUrqtUKpGeno758+cjMzOTjgvzzm00GnHDDTfg4MGDcLlcIIQgJSUFWVlZyMjIQGlpKU3xPv74Y0qSmkjk5ubi3nvvxb/+679SnnD+wNzT04OGhgYcP378s5Gi8AcJgUAAvV6PW2+9FTk5OVAqlbDb7XjmmWdw4sSJhFG6SaVSylEoFAqhVCpBCIHdbkdBQQGWLl2KkpIScByHtrY2HDlyBMPDw1f8eOU/VKmpqUhPT4fRaERvby8kEgnEYjHEYjFYlkV+fj6+//3v4/HHH6fOMrl0KBQKEQgE0N/fD6/Xi927dyc0B2dZFi+++CKKi4vBsix8Ph89WwUCAezcuRP79++PejqaMAfnH5kikQgKhQImkwlyuZxyWbe0tCSE2JJ3lIULF1J5DeDvm0RyuRz5+fnIzs6GRCKBy+Wi5cPJh9DLBX9f7HY7uru7IRKJ0NTUhMbGRpjNZhQXF8NsNkOhUMBoNNKD8Lm/g8/DtVotUlNTKTlqIqI4f+Dl5/qBsyScvNCA0+nEwMAArFYrOI6Lakqa0Bycb3NrNBraVnY6nejp6UnYdJ5AIIBKpcL1118Pu92OU6dOTRGMTU9PR0lJCXJycsCyLEZHR9HS0hLVs0IkEkF/fz9sNhsl1xeJRNDr9bjnnntQXl6OtLQ0JCcnTymrTa6P8618uVyOtLQ0iESihDm4UqlESUkJLb9GIhG4XC7Y7XY4HA5YLBa4XC5wHAeNRkPJQ6Oh1ZPQFEWhUCAnJwdFRUUQCoUIhULo6urC66+/njD2q3nz5uGpp55CUlIS6uvr0dnZid7eXjoeu3z5cqxYsQKpqangOI42gKKtLcQ7wWTu8f7+fjQ0NNADpkqlwrJlyyASiRAIBOB2u2kD6vbbb8f69ethNBoxf/586PX6hHGZFxQU4IUXXoBerwdw9oNos9loIAsEArjrrrug0+kgFovR09ODHTt2YOvWrVdM8ZcwB2dZFikpKUhLS0Nqaiolmu/s7ERdXV1Ccu8lS5bgmmuuQVlZGRWo4ruogUCAirCKRCKIRCKawsSSd/vcag5/X3w+H7xeL6qrq+ljnRfU4jgOPp8P8+bNg9FohFQqxUMPPYQdO3bgo48+imu5UCKRQKlUQqfT0YNuKBRCd3c3LBYLAoEAsrKykJmZCa1WC6FQCLVaTZ9YzzzzzBX5QlwcfPIjFPj71GBaWhrMZjMMBgMEAgEsFgt6e3vR29sb99o3y7JYvHgxVqxYgfT0dNTX12NiYgIjIyPweDwIBAL0gMcf5PgDUiLKmDwfIS+DeC5GRkbQ09OD/Px86PV63HTTTbDZbNi/f3/cxh6As+XOyV1VPo2y2WwIBoOQyWTIysqi3WA+rZJIJFCr1Xj++ednroPzFQH+4DS5ciKXy7F8+XIUFRXBZDJBKpWiu7sbHR0dCXFupVKJTZs2Yc2aNWAYBrt27cLHH3+MoaEhKtGiVquRkZEBg8EAtVoNv98PuVxONXtmEiKRCPbu3QuWZXHnnXciLS0NhYWFWLhwYVx7C36/Hy6XCzabjTbFGIaB0WiEXq+HwWCA2Wym8iaT+dlbWlqu2M6Y7mROPvTwn15+xjolJQVFRUVU1MlqtWL37t04ePBgLE36BCQSCTIyMvDd734XWVlZ8Hg8aGtrw1/+8hfU1tbSUhYfWYqLi6FQKOgQE38YjpX0+JXg5MmTqK2tpSmU0WhEWVnZlI5oLMGyLPR6PZKTk6FUKqlYgs/ng1KppN/jD8D8U8nr9aK9vR179uy5YgePCzfh5DefH9znD21yuRyEEPT396O7uxvDw8OxNonawSsGFxYWYsWKFVAoFBgbG8ORI0fQ3t4Om81G0yqZTAatVks14/myodPpTNj46adhaGgIPT09sNlsSE5Opls+sfwwntvfWLp0KcrLy6coHTMMg6SkJKjVairxbbfbKTm+w+FATU0N6uvrZ76DA1PnOvhWt1gshlqthlgspnqTNpstLgcg/kNWUlKCO+64A6tWrYLRaAQhBPX19Xj++eenODdwdh577ty5UKvVtOIjFArhcrnOO6syEzA6Ooru7m60tbVBoVCA4zio1eqYRHA+HeX/qVQqUVFRge9973u0IcbX7GUyGXzlavgAACAASURBVAoLC+l/E0LQ2dmJY8eO4Q9/+AO6urrgcDiiUvGJaxWFlxkMhUL0gCEUCuF2u9Hc3By3EpZQKIROp8N3vvMdqv0eDofx17/+FYcOHUJ/fz89BzAMg9TUVNx///343Oc+B4FAgEgkQtXCWltbMTIyMmPJiCKRCLxeL9xuNyYmJmIm1ahQKKDT6SCVSlFQUICSkhI8+OCDNAWd/NSY/AHzeDw4duwYnn/+edTW1tLqVbRsTEiZkHdyvhrBzyLEayWNd8SUlBTaYLLZbDhz5gyampro7ItKpUJycjI2b96M5cuXw2w200MQn38PDQ1hYmJiRjo3r5/Jl9/4fddYIBgMwu12IxQK0bSNv+65a3SBQAB/+9vfYLPZMD4+jo6ODjQ0NGBsbCzqdiWsDs6vpQkEAoTDYXR2dkZ1TPJi4FWKAdB5E6/Xi87OTvT390Mul4NhGDq09J3vfAcKhYLqq7vdblitViryNBMlWfhGWlJSEtLS0ugOp0gkisn1eMEuftRBLpfD6/VOUYXmiw4OhwP/+Z//iZaWlpg49WQkxMGTk5OxaNEi6jR+vx9WqzVu2yZ8irFz505s2LABJSUlyM/Px+OPP47x8XGoVCranDAajVCpVLQCYLVasXfvXlRWVuLdd9+FzWaLe/7Nt+EnM+6Gw2H6dd7BHnvsMaxatQp6vR52ux0HDhzAz372s5g20QghGBwchEQiwdjYGM6cOYPa2lq8/PLLsNvt9F55vd64nLcS5uALFiwAx3Hwer2YmJiAx+OJa4ctGAzi/fffR2ZmJnJzcyGVSlFYWEgbOvx2Pz/vHQwGMT4+ju3bt2P79u1oaWmZ8obFEzwpUlpaGsLhMBQKBfR6PYqKiuguq0gkwrx585CSkgIA2Lt3L44fPx4XdbtwOIyRkRE888wzNJUbHh6mUTyeSIiD85wYfHloeHiYChTFC+FwGNXV1Th58iTy8/Oh0+loKY1fbAiFQvB6vRgdHYXH48Hw8DD27t2L6urquHO0TMbklTS+I1xQUIB169YhJSWFPnE8Hg+8Xi96e3tx8OBBNDQ0xGUEghCCiYkJ/OlPf4r5taZlzHRfAEg0XnfffTdpaGggoVCIfPDBB+Sxxx6Lyu+9XDvFYjEpKCgge/fuJV6vl0QiEeJ0OklzczPZtm0bKSgoIBqNhnAcl1A7J7+EQiHR6XRk8+bN5Kc//Sk5evQo8Xq9JBgMkkAgQNxuN6msrCQvvfQSKS8vJxKJ5HKuczza732sXhfy2bhHcP7Rz4/GHj9+HHv27Im3GVMQCATQ29uLb3/721CpVHR4iW8z9/b2Jmzm5ELgqxVHjhxBU1MT9uzZg40bN8Lj8cBut6Onpwf9/f2wWCwYGBiYMdv08UbcHZxhGDidTrS3t6OpqQn19fXo7++PtxlTQAih3CFXC/hzwcjICGw2G/r7++kCxsTEBPr7+2G32/9hHZtH3Mk3RSIR8vLyMG/ePNTX12N4eBh2u/1Kfy0AXDXaN1eLnYgh+Wa0cSHyzbg7OM8ey9eeeaq2aOBqcZyrxU58Bhw87ikKP7QfCARmBE/HLD7bSIiDT/7nLGYRS8wKwcYfV4udwNVja+aFvnFJOfgsZnG14ZIi+NVy0Ji1M2q46pWOZ7SM4CwSjpmakkwbsw7+D46ZuEsaTcw6+D8oeJbXWQefBQDQOetoKxAkAiqVChkZGcjNzU20KTFHwhUezofzOVCiqj38Eu3k9bpgMIhwOHxVNqoEAgHy8vJQXFwMnU6XaHMATOXPifY9nTEOzlMCL126FF/96lchkUjAsiyCwSB27tyJ999/HzU1NXFzKo7jsHTpUlx//fXo6uqC3W6HxWJBVVXVFK3MqwUCgQDr16/HQw89hJKSEhw5cgSHDx9O+N+RmZmJP/3pT4hEIjh16hSee+45DA8PR40TPmEOLhAIcOONN8JgMFDKrpSUFOTn51MyToZhEA6HMTExgerq6rimBrfffjuWLFmC0tJSNDY2wmq10o30RDvFpYBfjuBZu8rLy2EymXDo0CGMjY0l7G/hOA4rV67EmjVrUFRUBEIIZDIZHnroIYyOjtIx32PHjl0R01lMHJxhGIhEoilOyi+e8pIbCoUCX/3qV1FSUgKz2fyJ3zGZBrigoIDySscDAoEAX//611FQUAAAlEE2Hute0QTLspBKpTAajXj88ceRn58PhUJB1++ioUZxOWAYBlqtFl/84hfxwAMPUI3P+fPno6ioCF6vF01NTTh69Cja29uvbOw32hs9DMOQlJQUsmXLFlJdXU06OjrI3/72N/L000+TzMxM8tvf/pbU1tYSp9NJQqEQiUQihEcoFCJ+v5+43W7i8XiI0+kkFouFPP/882TJkiVx2ZRRq9WkoqKCdHR0EJvNRurq6oharSYsy0Z9+yRav+9874FIJCILFiwgjz32GKmvrycej4eEQiHi8/nI9u3bycaNG6ezoRSTjR61Wk36+vqI3++n7304HCY+n49YLBbidDqJ0+kkVquV1NbWkjvvvPNTbY3bRg/HcSgtLUVBQQEyMzOhVCohkUiQkpKCnJwcVFRU0LQEAF1K3bZtGwghSE5ORkVFBYxGIyKRCCYmJrBjxw709MSn55CdnY0nnngCSqUSp06dwvvvvx+3DfArgVQqxQ9+8AMIhUJ4vV40NzejsLAQhYWFMBqNYFkWVqsVPT09+NWvfoW6urqEbCjl5eVhxYoV0Ol0EAqFlAf9tddeQ0tLC3w+H4RCIVQqFYxGIzZu3IicnBwUFhbizJkzl3y9qDo4T4lWWlqK9PR0aDQaiMViyGQymM1mlJWV0f+XEIKxsTEMDw+jtbUVf/zjH6FUKlFUVISioiLo9XoEg0GqshAt7fKLISUlBQsWLMCmTZtgs9nQ1NSEjz76KGEiWJ8GXu2Nl4B5+OGHwXEcrFYr/vznPyM3NxcGgwF+vx8+nw/Dw8M4ffo0du7cGTcOmnORlZWFG264gTKaORwOdHd346233qICVLxIbH5+Pm677TbMmTMHCxcuRGNj4yUHmqg6OE9nsGnTJuTm5l5QPx04SxTz3HPPYf/+/WhqagLDMLjttttQXFyMvLw8yGQy9PX1ob29PW4b98899xzWr18PlmXR3NyMM2fOoKOjI+bXvVTw9XihUIjNmzfjxhtvxC233AKBQACbzYbR0VG89dZbYBgG6enp2LBhA8rKynDq1Cl8+OGHCV1jy8/Px+233w6GYXDgwAHs2rUL//M//zPl/SX/Jytos9mg1+txxx13YN68eXjnnXfg8/kuyRei5uAMw1B1r8HBQSpHAQBOpxMOhwODg4NUgSASiaCmpgZutxvZ2dn4/Oc/j3Xr1iE3NxcKhQKEEFRXV+OFF16Im9qDwWCAwWAAIQS/+MUvUF1dHZfrTgcikQhGoxHf+MY3kJmZSck0U1JSkJSUhEgkgtbWVvzlL3/BgQMH0NvbCwCwWq2w2WzYs2cPRkZG0N7enpB0i2EY/OxnP8PKlSsRiUTwy1/+Env27EFtbe157eHp8UKhECQSCaRSKYRC4SX7QlQjOM/vXFVVhdHRUSQlJYEQApfLBYfDgZGREWRnZ1M+QJZlkZ6eDpPJhLVr16KkpARarRYMw6CmpgbHjh1DfX19zN8QlmWhVquhUCio3s2ZM2cwMDAQ0+tOxy5ejyczMxPz5s3DmjVrkJWVBZlMhlAohImJCdhsNjQ3N6OhoQEHDhzA8ePHqYoDn5709vbC4/EkhGZOKpXCbDZj9erVSE9Ph81mw969e3Hy5En09fWd92f4hprT6aSB8nyKcp+GqDk4v4o2OjqKF1544RPf50uHP/3pTzFnzhykpqaioqICmZmZmDt3LpYtW0ZVuEKhEH70ox/h1KlTcYk2YrEY8+fPn1JCc7vdCc29WZaFRCKBXC5HUlIS7r77blx//fXIzs6GSCSCz+eDzWZDTU0NqqqqcODAAbS1tU2ht+BLtC6XC+Pj4wn7W0wmE77whS8gNTUVfr8fDQ0N2LVr16dqG0UiEfT19YFhGJqaXGqZNm6NHp5V9Nlnn0VpaSmWL18Oq9WKefPmwWQyUZb/kZERHD16FA0NDRgdHY2LbUajEf/7v/8Lk8mE6upqfPnLX05Y9OZlDH/1q18hNzeXKiNIpVIAQF1dHVpbW9Ha2ooDBw5gbGyMcpSfS43Gl8oSyeeyefNmrF27Fps3b4bVasX777+PV1555VMPuXwlJTc3FxKJBCMjI+eVFfw0+Zi484OPjY2hubkZDMNgzpw5UKvVU+QtBgcH8dFHH8WN989gMCAvLw8ZGRngOA4ej+eiIlix5gE3mUy4+eabsXDhQphMJkgkEkoj5/P5qMrE6OgoFRTgI71arZ7Co+hwOKKiNXm5YBgGCxcuxOLFi6HT6bB161YcOHAA/f39n2pTeno61q5dC6VSSSP3+X7m06J6Qlr1POXwihUroNVqaXTy+/3o6enBu+++GzclsOzsbCxcuJDKApL/0+OZ7Mh8HszTPfPyJTxLbTQdKD09HY899hjMZjNN2XhKYolEgpycHEqw2dPTg66uLgBnJVZycnJoXbm1tRU9PT2YmJiA2+2O+8GS1zRauHAhSkpKEA6H8frrr6O1tXVa96uwsBD33HMPZDIZXC7XBQPOpwXBhDg4T/ebk5MDmUwGv9+PUCgEl8sFp9MJp9MZtzdk06ZNePjhh0EIgc/nQzAYhFgsprksy7JYuXIl7r77bqxevRpJSUlgWZZKeN97771RZeZqbm7GN7/5Tbz00kvIyMigpT/grNPwpPwGgwHLli2jXwfOfhD5SOf3+zEyMoLKykr827/9W1ylGTmOQ2pqKl544QXMnz8ffr8f/f39cDqd0y5Rms1mVFRUQCAQoK2tDVVVVZdFNZKwYatQKITa2lpaKUhKSqLiVCqVCg6HI+ZOzkdGhmHgcrnwu9/9DgcPHqTE7QsXLsTq1atRUVGBwsJCmM1miEQicBxHy1Y//vGPcerUKRw/fhzV1dVXnFa5XC40NjbimWeeoUNovAhtUlISbr31VqoVxEf480EsFkMgEGDFihX44Q9/iB/84AcYGhqKuZMLBAIUFxdjyZIlWLx4MTQaDSYmJi6J2Xb9+vUoLS2lWqTHjh3Dhx9+eFlPyoQ5eDgcRkNDA7RaLVQqFebMmUMdXKlUwuVyxaU8aLPZ0NXVhXA4jPfeew/Hjx+H3++HTCZDfn4+Nm/ejAULFiAcDtNyG/+hFIvFuOuuu1BQUAClUgmbzQaLxQKn03nZtftAIICRkRG8/fbb9Gu8cJPJZIJer0c4HIZer6eR+1wyfOBs3Tw5ORm5ubnIzMzEiy++CIvFElMHZ1kWWq0WZWVlWLt2LbKyskAIgcPhwOnTp6d1TxiGwTXXXEMnDL1eLxobG1FTU3N5qWAi6JMBEJZlSWZmJrn99tvJT37yE2KxWMjExATZvXs3KS4uJiKRKOZDTFKplGRkZJC5c+cSkUhE/m9znAAgAoGAPPDAA6S9vZ34fD5SX19PXnvtNSKVSolEIiGpqank1ltvJX19fSQcDpNwOEzGx8fJs88+S0pLS2M6bMUwDGEYhrAsS4RCIVEoFMRgMBCDwUAUCgWRy+WkpKSEHDhwgHi9XuLz+ci6deuIwWC41GtNe9iKZVkil8vJww8/TPbu3UuCwSAhhBCHw0E+/PBDkpKSMi36aYFAQJqbm0k4HCbBYJAcPnyY3HbbbdN+72M+bHUp8Hg8lA3V4/FAJpMBOCvOGo/Z70AggNHRUXAcRyMbLzP+5S9/GevWrUNycjIYhoHD4YDVaoVOp4PH40F6ejruvPNOSjbPa+LceeedKCwsxPe+9z3aXIk2JrOD8ecZPjryKdJkzSO+9R3LjrBWq0VeXh4effRR5OTk0GWV119/HXv37p2W1Etubi7uu+8+6PV6Wvt+6qmn0NjYeNl2JcTBJ+sjOp1OWCwWDA4Owmg0IhgMUrnnWIOvhIjFYhQXF0Mmk0EqlUIul2P9+vUoKiqCTCYDIQRyuRxGoxFr1qyBw+FAdnY2ioqKaFWFH1XIzs6GTCbDDTfcgJqaGvT29sacHpr/OyaDrzzwH4ZAIBDT9CQtLQ1r1qxBYWEhvWculwu1tbU4derUBQ+XDMNALpejoKAAixYtwrp162jhwWKxoLa29ooG7eLu4CzL0hdfIWhtbUVlZSXKy8vhdrtpbhkPW9RqNTIzM/GNb3wD+fn5SElJgdls/sSgWFFREfLz87FhwwZ6ABaLxdRp+OEn4Gzj6D/+4z+wbds27N69G7/97W8TUovmozsA2giKFZYuXYrvfve7kMvlAM5+oAYGBtDa2krnYiaDD2BisRhZWVl47rnnUFFRQeeQBgcH0djYeMXd5Lg4OO/MEokEaWlp8Hg8sFqt8Hg8VPSzp6cHZrMZLpcLarU65k7OT+Jt2rQJy5Ytg0ajoRUSgeCTt4VfOpZIJFAqlRgcHMSePXtw9OhRDAwMYGxsDEqlEsBZx+Kn4RwOR8JEYnnZwMkSfrECX1niHZc/iJvNZphMJoyMjNCyq0wmw9NPP43S0lJkZGRAJBJBr9fTnz9y5Aj+8pe/4A9/+MMVa6fG1MH5+fDFixfDZDJBqVSira2N1r35COP1euHxeCCRSCAQCKDT6WIiNz0Z/JyD3++HSCRCKBSiMnznAy9e63A4qGzIyZMn0dzcTB2ZHwqKRCIIBAL0w5sI5+Y31XlHu1AnMFoYGhpCdXU1VqxYQe+jRqPBTTfdhMLCQiqWy8u4X3PNNcjKyoJerwfw9/eDnzA8ceJEVFbqYurgLMtCoVDglltuwaJFi6DRaPCjH/0I/f398Pv99A/mb7xOp6OlplhH8HA4jMrKSixatAhpaWlQKBTQaDSQy+VUCJYHP0fDt/FffPFF1NTUfOIAGa/u66eBT71EIhF18FiXXNva2rBt2zYUFxdDqVRCIBAgOTkZ999/P7Xp3HOV3++H0+mk4wW1tbX4/ve/j7GxsaiNacTMwRmGgUQiQUlJCVQqFZxOJ7q6utDU1ITh4eGzJRyBACUlJVi8eDEeeeQRmM1mnD59Gvv374/Lxkk4HMbLL7+M3/zmN9TmTzvc8iPBM0mQ6lxkZ2fj6NGjdFGbH0GIZQRvampCZ2cnWlpa8JWvfAUbNmy4qKpyJBLB008/jddffx0ul4sOhfl8vqjaGfMIrlKp6KBQKBSCUqmkA/rl5eVYuXIlysrKYDKZ6Ml5cHAwbm3lYDA4Y1fSLgdisZiqNAsEAlitVtTU1MQ8YEQiEfh8PtTV1eGVV17B9u3bP/Up/PHHH8NiscT0/sfMwScPKvEVBoVCgezsbJqKrF+/HhUVFcjPzwfDMGhra0N7e3tCZ5evdiQnJyM7OxsCgQBOpxPd3d3Yt29fTOrx54IQgqGhoYTRUZwPMY3goVAIo6OjyM7Oxvz588GyLAoLC+mBjm/Ph8NhdHd3Y8uWLaisrIylSZ95PPTQQ3jggQfAMAw+/PBD7Nq1C1u3bk20WQlDXKooIpGIVkjMZjMikQgdQQ0GgxgYGMBPfvITNDc3/8PrOl4p3nvvPTQ2NiInJwcHDx6MG93GTEVMHTwcDmN8fBy1tbVwOp2Qy+VQKpXQaDTQ6XRUYbi+vh6VlZUYHx+f8fwjMx0dHR0YHR1FW1sbOjo64HQ6E21SQhFznUx+JJXnRlm2bBkqKiqwevVqsCyLbdu2YdeuXTh48OAVOze5SqRBrhY78RnQyYybECxfj5XJZCgoKEBZWRlOnjyJgYEBWK3WqJzyrxbHuVrsxGfAweM2i8IPBDkcDvT399NNDZfLNZt3zyJmSIgQrN1uR0dHBxwOx4xumMzi6kdChGDdbjfcbveV/ppzkTnp368W0dKZbCdw9diaeaFvzArBzuIzjVkh2ATgarETcRKCZVk2ahW0czFjNHpmMSMRk5SEZVmYzWZIJBKIxWIIhcLY1ewTtXQci9esnVF/RV3hgWEYolarya5du0hfXx9xOp2ksbGRrFix4opUNGbk0vEsogt+lt5kMmHjxo3o7u5GX18fGhoa4sIzcyHwyxdGoxHXXnstvvCFL6CsrIzyvshkMrpN9ZmVEUw0Ji9CX20Hb35zRyQSQavVIi0tDQsWLEBKSgqys7NhNptx8uRJWCwW2O32uNum0WhQUFCA3NxcrFu3DmvWrIFMJkM4HIbH40FPTw/8fn9MFs1nHRx/J+/nxUj5dbqrAfwoMj8Hzu9ACgQCLFu2DAaDAWazGU8//TT279+PqqqquNonFApRXFyMZ555BgsWLKB7lzyt88DAAN544w0MDAzE5J7POjhAV6b4CMITcM7kaC4SiaBSqbBgwQJYrVY6ZGWz2XD69Gns2LEDmzZtwurVq/H5z38ejz76KAQCQdwdfPny5Vi5ciWMRiNsNhtkMhkUCgX8fj/6+vpQV1eH9957DxMTEzFZcpl18Ek4l1ubx+RVtpkQ2TUaDfR6PZV7mZiYoJtJ/NjDxMQE3fZnGAZGoxEajSaudrIsi9zcXEoE1NXVRTlkWlpa0NfXh66uLoyPj192R9tgMFx0FzYhDi4QCOihggcfRfnt9USDd3D+gMQfhHi+bX4NLxERPiUlBXl5eVi6dCnq6+vh8/nOS6/gdrvh8XjoJvv56DBiBT7ty8/PR15eHgCgtbUVY2NjsNvt2LZtG8bGxq64o20ymTA8PHzB78fdwZVKJa677jrcc889WLx4Md3oGR0dxV//+ldUVVVhx44d8TbrokhKSsIPfvAD3HXXXRAKhXC5XDh8+DC++c1vwmKxxH2eJiMjA1KpFB999BEaGxsvuI5WXl6O8vJyAJ9OFB9t8AJZJSUlmD9/PjiOg9FohN/vh81mg9VqvWLOE4Zh0NDQcNGnalwcXCgUYu7cuXjwwQehVCqpLg8vUEoIgVKpxK233orFixfj5ptvxo9//GOMjY3Fw7yLgqeX++ijjzAxMYH77rsPHMchOzsbDz30EH75y1/GbYeU35DiD2j9/f3wer2feOLxekiFhYWYM2cOgNhTt50Lnoi/qakJer0e+fn5kEgkEAqFlC7kSsFv4l8McXFwk8mExYsX48tf/jLkcjlCoRC8Xi8GBgZACIFQKERaWhrKyspQWlqKUCiE//7v/4bFYpkRhzyv14vDhw+jr68PRUVF0Gq1EAqFuP7667F169a4OjjHcfD7/fD7/RgfHz+v0woEAhiNRmRlZcFsNgNAzMk3zwUhBB6PBwMDAxgaGqLapyKRiObh8eCfjLmDcxyHJ598EuvXr6fy3b29vaisrMS3vvUt+Hw+ZGVlYffu3dBoNBAIBAiFQpQqbaZQOng8HjQ2NmLz5s3QarVYvHgxvvvd70IgEIDjuJinKTw7Ac9pfj7CTeDs0zI5ORmPPvooUlNTKWVbR0fHRXPVaIO/rlKphFarhVwuR1paGiwWC/17rnoHF4lEmDNnDvLz82E0GhEKhfDLX/4Sx48fx4kTJ+gOZl9fHx5//HF8/etfR35+Pnw+H31DZ4qD8yCEUGoGPirF2sH5yM0vbxuNRiQlJWHBggX4+OOPEQgEIBaLodPpkJSURInyAdDvXYyWLlY2CwQCFBUVobCwkFLyMQyDwcFBKvIaa8R8q57XqpdKpQiHw+jp6UFTUxPOnDkDALSSwrNFBYNBuggxE9KT8yErKwu5ubnQaDQQCoUxdxyWZSEUCiEUCunhLS8vD6mpqVQQlpc40Wq10Gg0SElJmcIspVQqqdhXPMBxHHQ6HQwGAzQaDfUFfhE9XupvMedFGRoaouU0QghVCOMfT1KpFBkZGXjiiSeQnZ2NcDiM4eHhSxIsiicYhsGKFSuwcuVKAH9nnY0lRCIRxGIxddiCggKsXLkSS5YswQMPPAAAlAovFArB7/djeHgYKpWKcq1nZGTAYDDEjelWJBKhqKgIKpVqSjnY6/XCarXG/Po8Yk4bMTo6in379sHr9SIrK4tGoLlz58LtdmPt2rVYuXIlcnJyIBAIUF9fj6eeegpjY2MzLoKzLAulUgmDwQCFQgGn00lVkWN1PaFQiKSkJCrJzXEcNBoN5R0cGRmhnI+Dg4OwWCywWCxoa2vDli1bUFFRgZSUFEgkEixatAj/9E//hNdeey3mqZ9EIsHixYuhVqvp18RiMUpKSnDffffh+eefh1gshkQioUrMsTgEx/yQGQqFUFVVBZfLhYULF8Lr9SIlJQWrVq2C0+nE0qVLUVJSAqlUStUeeEnqmQReouSGG25ATk4OJBIJRkdHP6EuHE0oFArodDpKMx0IBGiTTCAQwG6345133kF7ezv6+/up+KvD4YDFYkFNTQ2USiUtx2ZmZmLdunXYunVrzB08EAigsbFxyoefEAKxWIyUlBTcf//9kMlkkEgk8Pl8U7QwCSFoaGhAdXX1FVPOxaVMeOjQIbS3t8Nut0OhUFCdep/Ph+LiYuTk5EAoFMJut2N4eBhjY2MzoiU+GSKRCCkpKXjwwQdRWFiISCSCU6dOxdRRNBoN8vPz0dLSQtvwfMPG5/Ohu7sbL730Evr6+s57vw4fPgyRSISFCxdCpVIhIyMDarUaWq0WoVAoprZ7PB7s27cP//Iv/0K/5na74fV6wbIsnnjiCajVakgkkk/8LCEEW7duRW9vL7q7u6/sAB+vhQeWZYlIJCIymYyUl5eTr3/962Tnzp2ku7ub+Hw+EggEyJNPPkmWLFkyIxcJ7rzzTrJ161bidDqJ1Wol27ZtI1qt9rKG9Kdjp0QiIRUVFeTxxx8nxcXFRKfTEY7jiFAoJCtXriTXXHMNUSgUU5Thzn1xHEcUCgXJysoiNTU1ZGJigoRCIVJXV0e+8IUvTMfWy154YBiGSKVSsnv3bhIKhYjP5yM//vGPyYYNG0hOTg75zW9+QxobG0koFCKRSIREIhGqrBYOh4nb7SZdXV0kLy+PyOXymb/wwKsehEIhdHd3gxCCway9XwAACDJJREFUVatW0ceSy+VCR0cHlaaeSeA7sUuWLAHHcfj973+PvXv3xnSJQCAQQK1WIy0tDfPnz4dAIEBfXx98Ph86OztpN/Ni4OetR0ZG8NOf/hS33XYb7rnnHmRkZFC5lViCJ1Xdt28fDh8+jF27dmFgYADj4+PYunUr9u7dSyssPFiWxTe/+U0YjUYYjUb88Ic/xAcffIDKykoMDAxcsg1xn0WJRCKUycrpdFIH5yX94nnCng5YlkV2djby8/ORk5ODQCCAXbt2Yf/+/TGtfYvFYirbXVhYCK/XC5/Ph4GBAYyOjk67hhyJROD1erFt2zakpKTg7rvvhlKpvCg5fTTAR9DGxkZ0dHTgrbfewuDgID1bHTp06Lw/JxAIcM0112Dx4sXIysrC3Xffjf7+fjQ3N18dDg78fYbA7XYjHA5DIBDQ0uFMy71lMhneeOMN5ObmUllt3tliCZPJhNTUVJhMJiqrotVq8bvf/e6y7hF/z4PB4Hnz3ljhlVdeASFk2hWSUCiEr3zlK3j00Ufx7LPPgmVZ+P3+y5aHiV9r6xxwHIeCggJotVoEg0G0tbXFhaT9UsEwDNXwGRwcxF133YW6urqYXzc9PR05OTnIy8tDaWkpNBrNFTdHmpub8fbbb9MoGqtWuUgkokzCfGp6Kbj99tuxePFihMNhDA0Nob+//7LHDBK28MAwDHQ6HS0TnThxAhMTE4ky54LgdeIBwG63Y//+/XEZH/D5fGAYBhqNBhzHIT09Henp6Vfk4F6vFxaLJaZPSX65WK1W02nQS7FZIBBgyZIlyM3NRTAYxJEjR9Dd3X3ZT8yERXBe4Zav577++uszSvqCB9+Gd7vdcW0xnzhxAr29vZBKpZBIJFi5ciXuuOOOy466/IclKytrympetCEWi7FkyRLcdNNNlzzCwLMPr169GgUFBfB4PHjqqaeuSPUjIQ7O6/XwrWSPx4PTp0/PGBk+Hmq1GllZWWBZFkeOHMGbb74Zt+6qy+VCe3s7jh49ikgkAolEAoVCAZlMdsmOw7IsXnvtNbzwwgtYvXo1tm7dioaGhqjbzIvr3nHHHbjuuusgFounbSvHcVi5ciWqqqqQnp6O999/HzfffDO6u7uvKHVNSIqi0WiQm5sLiUSCUChEGwAzjWnWYDBgwYIF4DgO7e3tOHHiRNyuHYlE0NrainfffRdLliyBRCKBTqfDV77yFRw4cAA9PT3TWghJT0/Hxo0bsXTpUhiNRoTDYezatSsm0iYcx6G8vBy5ubmQyWQoKyvDkSNHLppeqFQq3HLLLVCpVJg/fz5yc3OxZ88e7Nu3D01NTVfe0U4Es1VxcTH5xje+QdxuNxkeHiY7duy4aMNiuq9o27l69Wry61//mvj9fvK1r32NcBwXld97KXYqlUrS19dHfD4fbYhs2bKFrFy5kqhUKiKRSIhIJCJCoZAIhUIiFouJ9P+3d+4grWxRGP5NTEwy44xBDQrGkwf4Jk0KC7FNJ2IhWEgqwcZS0NLO3sJTqIUBCxEsLESbNBYSJD6jQiQJooSYYMxDMyGP2bc4JMjxHL14k8zcsL9qYJrF5t+zZ++91vq1WsIwDOE4joyPj5N0Ok3y+TxJJBLk8vLyX12c4BsXPQzDELfbTR4eHkgkEiGrq6vEZrMRtVpNlEpl+bKPZVnCcRzheZ7YbDZye3tLEokEyWQy5Pn5mUxMTBCTyfStMZVFZ6uBgQE4nU6o1WqcnJzg8PBQdolVAGCxWDA2NgaVSlWzCpTfSafT6O/vx9LSEpxOJ9rb27G4uIj5+XkUi0VsbGwgEAggFouhpaUFRqMRXV1d6OnpQXd3d/k3MBKJYHd3FysrK1U7rcrlclhbW8P09DRGR0cxNTUFnU6Hi4sLnJ+f4/7+Hg6HA7OzszCbzWX3Y61Wi/39fbjdbqyvryObzcrf6fgzNBoN9Hp9Oanm6OhIijC+pKmpCSzLSiLs97y+vmJnZwc+nw9arRZzc3Po6+tDQ0MDHA4HkskkBEEoN/9hWRZ6vR4sy0IQBFxfX8PlcpX936v1MSkWi/B6vbDb7eXJNTQ0BKPRiJGRESSTSZjNZphMJnAcBwCIx+P4+fMnjo+P4ff7K74Pk0TgKpUKDMOUq3lubm6kCONLlEplTYsEPsPj8cDj8QD4VS2v0+lgMBgwODj4YSOXz+cRDofx8vKCWCwGr9eL7e1tRKPRqq6UoigiFArh6uoKBoMBra2taGtrQ0dHBywWC1KpFBQKBZ6enhCNRiGKIsLhMLa2thAIBKpyyFBzgSsUCmg0GvA8D1EUZW2lXSoVA/DXGkgpmJmZgd1ux/LyMoaHh8vn9AqFAoVCAaFQCJOTkwiFQnh7e6tp3Pl8Hpubm3C5XAAAq9UKo9GI3t5e7O3tIR6Pf0gxruakq7nAGxsb8fj4iIODA2QyGVkmV71HFMVyg3a57BMIIfD7/VhYWADP8x8aKAmCgGAwCEEQJJuUpbEquejd3d2VfelrOY6S/KKkUikEg8Fy7rdcKVWGl8QtF4EDv8bw9PRU6jC+RBAECIJQs9YavyNJNmGpp8fZ2ZlsBV5K/MpmsxBFEYVCoWb1jJTKUTMj2PcwDAOO45BMJivacYlU2Pums7MTVqsVPM/D5/NV7HKk0nFWEWoE+x0EQUAul5Pt5rJEPB5HNptFc3OzZEss5b8hicDldCLxGaUWaYIgyH4yUv6MJEawVeLHu+eKxlnh4oaqxVkF/i+x/vjbC2oES6lrJMsHp1BqARU4pa6hAqfUNVTglLqGCpxS11CBU+oaKnBKXUMFTqlrqMApdc0/iw5rF1WgW2cAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 216x216 with 16 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plot_digits(gan.predict_gan_generator(batch_size=16))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "etw8X24pr0cr"
   },
   "source": [
    "Not too bad.  Many of the generated images look plausibly like handwritten digits.  A larger model trained for a longer time can do much better, of course."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "LTtjqIsnr0ct"
   },
   "source": [
    "# Congratulations! Time to join the Community!\n",
    "\n",
    "Congratulations on completing this tutorial notebook! If you enjoyed working through the tutorial, and want to continue working with DeepChem, we encourage you to finish the rest of the tutorials in this series. You can also help the DeepChem community in the following ways:\n",
    "\n",
    "## Star DeepChem on [GitHub](https://github.com/deepchem/deepchem)\n",
    "This helps build awareness of the DeepChem project and the tools for open source drug discovery that we're trying to build.\n",
    "\n",
    "## Join the DeepChem Gitter\n",
    "The DeepChem [Gitter](https://gitter.im/deepchem/Lobby) hosts a number of scientists, developers, and enthusiasts interested in deep learning for the life sciences. Join the conversation!"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "17_Training_a_Generative_Adversarial_Network_on_MNIST.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}
